Lazy Iterators
./img/joys-of-apex-thumbnail.png
Welcome back to the Joys Of Apex! You may remember from the footnotes of Sorting & Performance In Apex that I mentioned an article on iterators, which I first read in February while browsing the Salesforce subreddit. One thing that I dwelt on for some time after reading the article was how my clients might be able to use the power of lazy iteration - which differs from the eager iteration performed by the traditional "for" loops we use quite often in SFDC Apex development - to speed up trigger handlers. Were there performance gains to be had by delaying the evaluation of records in a Trigger context? I would also highly recommend the YouTube video that is posted in the article: this talk on lazy evaluations from GOTO 2018 is fantastic
I was impressed at the time with the object-oriented approach that Aidan Harding and the folks over at Nebula Consulting had undertaken when implementing the Lazy Iterator framework. The idea for this article has been brewing since then. Their LazyIterator
object uses the Decorator pattern to add functionality to the underlying iterators found on all Salesforce List
objects — reading through their codebase got me re-excited about working with collections in Apex. You may recall from the Writing Performant Apex Tests post that using iterators to page through collections is much faster than any of the "for" loop implementations!
I'd also like to thank Aidan for generously proof-reading the beta version of this, which led to a couple of great edits. This article is assuredly better for his input.
All problems in computer science can be solved using one more level of indirection. — David Wheeler
So-called "Lazy" evaluated functions have their actual execution delayed until a "terminator" function is called. It's common for lazy functions to be chained together using fluent interfaces, culminating with actions being performed when the terminator function is called. What can Salesforce developers writing Apex code stand to gain by learning more about lazy functions?
Fluent interfaces — or objects that return themselves during function calls — also tend to satisfy one of the prerequisites for Object-Oriented Programming; namely, encapsulation. While fluency as a property of functions/objects has become associated more with functional programming than with OOP, many developers are being unwittingly exposed to pseudo-fluent interfaces (and, as a result, functional paradigms) through the JavaScript collection API; it's not uncommon to see various filter
, map
, and reduce
calls being chained together when iterating through lists in JavaScript. It's also not uncommon for people to underestimate the performance implications that come with using these functions — in JavaScript's vanilla functions' case, our code's readability increases at the cost of performance:
1linkconst list = [1, 2, 3, 4, 5, 6, 7];
2linkconst doubleOfEvens = list.filter((x) => x % 2 === 0).map((num) => num * 2);
3link
4linkconsole.log(doubleOfEvens);
5link//output: [ 4, 8, 12 ]
Well-named (and encapsulated) functions are appreciated by developers because they are easy to understand. Even if you don't know JavaScript, you can look at the above code snippet and divine its meaning: given a starting list of numbers, take the ones that don't have a remainder when divided by two, and multiply those values by two. However, when you look at what that code is effectively doing, you might begin to feel differently about it:
1linkvar list = [1, 2, 3, 4, 5, 6, 7];
2linkvar evens = [];
3linkfor (var i = 0; i < list.length; i++) {
4link var num = list[i];
5link if (num % 2 === 0) {
6link evens.push(num);
7link }
8link}
9link
10linkvar doubleOfEvens = [];
11linkfor (var index = 0; index < evens.length; index++) {
12link var even = evens[index];
13link doubleOfEvens.push(even * 2);
14link}
Yikes. Turning back to Apex, you might immediately see the parallels between this example and how a typical Apex trigger handler is set up:
1linkpublic class AccountHandler extends TriggerHandler {
2link
3link public override void beforeInsert(List<SObject> newRecords) {
4link List<Account> accounts = (List<Account>)newRecords;
5link
6link this.trimAccountName(accounts);
7link this.formatAccountPhone(accounts);
8link //etc ...
9link }
10link
11link private void trimAccountName(List<Account> accounts) {
12link for(Account acc : accounts) {
13link acc.Name = acc.Name.normalizeSpace();
14link }
15link }
16link
17link private void formatAccountPhone(List<Account> accounts) {
18link for(Account acc : accounts) {
19link // do stuff with the phone
20link }
21link }
22link}
This pattern is pretty typical. It's not uncommon in larger organizations for the big objects — Leads and Opportunities in particular — to have dozens (if not hundreds) of trigger handler methods, each of which likely involves sifting through the entirety of the old/new records prior to performing business logic. As trigger handlers grow in size, related functionality is frequently broken out into other classes; this tends to obscure just how much processing is occurring as a handler churns through records.
Once records are being updated, many developers have utility methods designed to compare the old and new objects handed to SFDC developers in the form of Trigger.old
and Trigger.new
lists, but even these utility methods frequently take the form of iterating over the entirety of the old and new records to isolate matches. (This becomes a problem when you need to isolate many different groups of changed records to do further processing; e.g. Accounts that need their Opportunities updated, Accounts that need their Contacts updated, if a particular Opportunity Line Item is added, go back to the Account, etc ...) So what can we do? Should we do something to address this "problem" — or is it really not a problem at all, but one of the results of a growing system? As usual, to answer that question, we're going to have to write some tests.
These particular tests will make use of Queueable Apex. Why use Queueables over our typical Apex tests? There's an argument to be made (by some) that the overhead introduced by the test classes themselves actually obscure the production-level performance results. I haven't found that to be the case, but with long-running operations, it's possible for running tests to time out, so we'll be avoiding that issue altogether. In order to avoid hitting any kind of Anonymous Apex timeouts, we'll choose the easiest of the async Apex implementations to rig up a timer:
1linkpublic class QueueableTimer implements System.Queueable {
2link private Datetime lastTime;
3link
4link public QueueableTimer() {
5link this.lastTime = System.now();
6link }
7link
8link public void execute(QueueableContext context) {
9link Savepoint sp = Database.setSavepoint();
10link List<Account> accounts = this.getExampleAccountsToInsert();
11link this.log('Starting execute after gathering sample records');
12link
13link insert accounts;
14link
15link this.log('Ending');
16link Database.rollback(sp);
17link }
18link
19link private List<Account> getExampleAccountsToInsert() {
20link List<Account> accounts = new List<Account>();
21link //savepoint usage consumes a DML row ...
22link for(Integer index = 0; index < 9998; index++) {
23link accounts.add(
24link new Account(
25link Name = ' Testing' + index.format() + ' ',
26link Phone = '8438816989'
27link )
28link );
29link }
30link return accounts;
31link }
32link
33link private void log(String startingString) {
34link System.debug(
35link startingString + ', time passed: '
36link + this.getSecondsPassed().format()
37link + ' seconds'
38link );
39link this.lastTime = System.now();
40link }
41link
42link private Decimal getSecondsPassed() {
43link return ((Decimal)(System.now().getTime()
44link - this.lastTime.getTime()))
45link .divide(1000, 4);
46link }
47link}
This code is going to interact with our extremely plain AccountHandler
object, called by the trigger on Accounts:
1linkpublic class AccountHandler extends TriggerHandler {
2link public override void beforeInsert(List<SObject> insertedRecords) {
3link //no methods for now
4link }
5link}
Right now, with the beforeInsert
method empty, kicking off the QueueableTimer
prints the following:
1link$Starting execute after gathering sample records, time passed: 0.788 seconds
2link$Ending, time passed: 32.035 seconds
It's important to note that the baseline is being measured prior to the Savepoint
being rolled back. Something that should be immediately obvious in looking at these results? It's not the setting up of the savepoint or the Account list that is leading to any kind of slowdown; this is just how long it takes for Salesforce triggers to process large record sets in 200 increment batches. I tested various other org structures, including those with Duplicate Rules enabled, workflows, validation rules, etc ... while those definitely slow down the resulting insert (Standard duplicate rules for Accounts added nearly 7 seconds to the processing time), the vast majority of the time was simply in creating that many objects. This should give you a good idea, per 10k records, how long it takes at baseline to process records: .0032 seconds per Account. Not too shabby.
We'll just update the AccountHandler
object so that the beforeInsert
method contains some vanilla processing methods as I had shown earlier:
1linkpublic override void beforeInsert(List<SObject> newRecords) {
2link List<Account> accounts = (List<Account>)newRecords;
3link
4link this.trimAccountName(accounts);
5link this.formatAccountPhone(accounts);
6link}
7link
8linkprivate void trimAccountName(List<Account> accounts) {
9link for(Account acc : accounts) {
10link acc.Name = acc.Name.normalizeSpace();
11link }
12link}
13link
14linkprivate void formatAccountPhone(List<Account> accounts) {
15link for(Account acc : accounts) {
16link this.formatPhoneNumber(acc.Phone);
17link }
18link}
19link
20linkprivate String formatPhoneNumber(String phone) {
21link if(phone.length() == 10) {
22link return '(' + phone.substring(0, 3) + ') '
23link + phone.substring(3, 6) + '-'
24link + phone.substring(6);
25link } else if (phone.length() == 11 && phone.substring(0) == '1') {
26link return this.formatPhoneNumber(
27link phone.substring(
28link 1,
29link phone.length() - 1)
30link );
31link }
32link
33link return phone;
34link}
Two simple methods — let's see how much processing time that consumes:
1link$Starting execute after gathering sample records, time passed: 0.403 seconds
2link$Ending, time passed: 33.469 seconds
That's a 4.47% increase in processing time. With smaller number of records, of course, such increases are hardly noticeable — until they are. I think anybody who's converted a lead at the start of a greenfield project versus several years into the implementation can attest to the fact (which I have cited previously in the React.js versus Lightning Web Components post) that delays as small as 50ms can both be detected and negatively impact the end user experience.
Somebody recently asked a question regarding the performance and organization of Apex triggers as they grow, and possible design patterns for cleaning up complicated handlers. Since it's something I've spent quite a bit of time thinking about as I pondered the LazyIterator
framework, I directed them to read the Nebula Consulting post on Lazy Iterators. Their response?
That seems hard for the next guy to learn, to be honest
They're not wrong. Looking at the Nebula Bitbucket shows that a lot has changed since the article was written last year; many updates to the framework, and a lot of potential. But, like FFLib, the issues with larger frameworks lie in stimulating adoption. How can you get new users to grok the intent of your code, particularly with very abstract examples? Documentation helps, but it's not perfect, and as soon as you have documentation, you're in an arms-race with yourself to keep it up-to-date. Typically, frameworks achieve widespread adoption by providing some combination of three things:
These tenets hold true for domains outside of programming, of course, but across the tech stack it's easy to see different permutations of this concept:
In many ways, I'm the ideal consumer of the LazyIterator
framework — I'm a consulting company looking to bring performance improvements to my clients, many of whom already deal with routine system slowdown due to growth. How can I wrap the functionality presented by the LazyIterator
concept into something that's easier for others (including myself) to use and understand? The examples that follow are heavily-influenced by Aidan Harding's work. I re-implemented everything from scratch — not something that I think is necessary the vast majority of the time, but somethign that I think indeed helps when looking to further your understanding of new concepts.
This is a stirling use-case for inner classes. Much like how people got really excited when IIFE's became a big part of JavaScript development, inner classes allow you to hide the scary parts of your code from other consumers. And, much like IIFE's, they can abused/over-used. They're not always the answer! Another alternative, which is frequently talked about in the book Clean Code, is the Adapter pattern, where you isolate the code foreign to your codebase through the use of interfaces and boundary objects.
Since we're talking about trigger handler frameworks, I am going to tailor the code that follows towards exploring how to achieve lazy iteration in a trigger handler's context. It should be noted that the LazyIterator
framework covers an enormous swath of material and use-cases; keeping it simple here will help to keep the overall size of what you're reading down to a manageable level.
I'll start "simple", with a wrapped iterator capable of detecting when SObjectFields have changed:
1link//separate file for the interface
2link//because outside callers can
3link//(and need to) implement
4linkpublic interface BooleanFunction {
5link Boolean isTrueFor(Object o);
6link}
7link
8linkpublic class ObjectChangeProcessor {
9link private LazyIterator iterator;
10link
11link//assumes objects are in the same order
12link//as in Trigger.oldRecord, Trigger.new
13link public ObjectChangeProcessor(List<SObject> oldObjects, List<SObject> newObjects) {
14link this.iterator = new LazySObjectPairIterator(oldObjects, newObjects);
15link }
16link
17link public ObjectChangeProcessor filterByChangeInField(SObjectField field) {
18link return this.filterByChangeInFields(new List<SObjectField>{ field });
19link }
20link
21link public ObjectChangeProcessor filterByChangeInFields(List<SObjectField> fields) {
22link this.iterator = new LazyFilterIterator(this.iterator, new FieldChangedFilterProcessor(fields));
23link return this;
24link }
25link
26link public ObjectChangeProcessor filter(BooleanFunction function) {
27link this.iterator = new LazyFilterIterator(this.iterator, function);
28link return this;
29link }
30link
31link public List<Object> toList(List<Object> toList) {
32link return this.iterator.toList(toList);
33link }
34link
35link//BASE LAZY ITERATOR
36link virtual class LazyIterator implements Iterator<Object> {
37link private final Iterator<Object> iterator;
38link
39link public LazyIterator(Iterator<Object> iterator) {
40link this.iterator = iterator;
41link }
42link
43link protected LazyIterator() {
44link//one of the more fun statements
45link//made possible by self-implementation ...
46link this.iterator = this;
47link }
48link
49link public virtual Boolean hasNext() {
50link return this.iterator.hasNext();
51link }
52link
53link public virtual Object next() {
54link return this.iterator.next();
55link }
56link
57link public List<Object> toList(List<Object> toList) {
58link while(this.hasNext()) {
59link toList.add(this.next());
60link }
61link return toList;
62link }
63link }
64link
65link//Wrapped SObject Pair Iterator
66link virtual class LazySObjectPairIterator extends LazyIterator {
67link private final Iterator<SObject> oldIterator;
68link private final Iterator<SObject> newIterator;
69link
70link public LazySObjectPairIterator(List<SObject> oldObjects, List<SObject> newObjects) {
71link super();
72link this.newIterator = newObjects.iterator();
73link this.oldIterator = oldObjects.iterator();
74link }
75link
76link//wrapper POJO
77link private class SObjectWrapper {
78link public final SObject oldRecord, newRecord;
79link public SObjectWrapper(SObject oldRecord, SObject newRecord) {
80link this.oldRecord = oldRecord;
81link this.newRecord = newRecord;
82link }
83link }
84link
85link//realistically, you could just do one of
86link//these, since it's required that
87link//both lists have the same # of elements
88link public override Boolean hasNext() {
89link return this.oldIterator.hasNext() &&
90link this.newIterator.hasNext();
91link }
92link
93link public override Object next() {
94link return new SObjectWrapper(
95link this.oldIterator.next(),
96link this.newIterator.next()
97link );
98link }
99link }
100link
101link//Iterator that allows for filtering ...
102link virtual class LazyFilterIterator extends LazyIterator {
103link private Object next;
104link private final BooleanFunction filter;
105link public LazyFilterIterator(LazyIterator iterator, BooleanFunction filter) {
106link super(iterator);
107link this.filter = filter;
108link }
109link
110link/* NB: the Nebula version uses
111linkanother method, "peek()", but for both
112linkterseness and expressiveness, I find this
113linkrecursive method more descriptive: hasNext()
114linkpeeks values ahead of the current object
115linkin the list for matches, advancing the
116linkinternal iterator's place till it
117linkfinds the next match or reaches the end.
118linkThis is tail-recursive and, as such,
119linkstack safe */
120link public override Boolean hasNext() {
121link if(super.hasNext()) {
122link this.next = super.next();
123link return this.filter.isTrueFor(this.next) ? true : this.hasNext();
124link }
125link
126link return false;
127link }
128link
129link public override Object next() {
130link if(this.next != null && this.next instanceof SObjectWrapper) {
131link return ((SObjectWrapper)this.next).newRecord;
132link }
133link return this.next;
134link }
135link }
136link
137link class FieldChangedFilterProcessor implements BooleanFunction {
138link private final List<SObjectField> fields;
139link public FieldChangedFilterProcessor(SObjectField field) {
140link this(new List<SObjectField>{ field });
141link }
142link public FieldChangedFilterProcessor(List<SObjectField> fields) {
143link this.fields = fields;
144link }
145link
146link public Boolean isTrueFor(Object obj) {
147link SObjectWrapper wrapper = (SObjectWrapper)obj;
148link Boolean hasMatch = false;
149link Integer counter = 0;
150link//since the matching variable is also
151link//what's being returned, I prefer this format
152link//to the usage of a "break" statement
153link while(counter < this.fields.size() && !hasMatch) {
154link hasMatch = wrapper.oldRecord == null ||
155link wrapper.oldRecord.get(this.fields[counter]) !=
156link wrapper.newRecord.get(this.fields[counter]);
157link counter++;
158link }
159link return hasMatch;
160link }
161link }
162link}
As always, usage of the Decorator pattern means you're looking at a lot more code. However, I've minimized the usage of standalone custom classes in this version, using the ObjectChangeProcessor
to wrap everything up with a bow. For more generic usages, you probably wouldn't wrap the LazyIterator
itself. What does all of this code get us? Easy and lazily-implemented detection of records in a trigger that have changed based on field conditions:
1linkprivate class ObjectChangeProcessorTests {
2link @isTest
3link static void it_should_correctly_filter_records() {
4link Account acc = new Account(
5link Name = 'Test Account',
6link NumberOfEmployees = 5
7link );
8link
9link Account newAcc = new Account(
10link Name = acc.Name,
11link NumberOfEmployees = acc.NumberOfEmployees + 2
12link );
13link
14link Account accTwo = new Account(
15link Name = 'Test Two',
16link NumberOfEmployees = 5
17link );
18link
19link Account accThree = new Account(
20link Name = 'Test Three',
21link NumberOfEmployees = 6
22link );
23link
24link Account accThreeNew = new Account(
25link Name = accThree.Name,
26link NumberOfEmployees = accThree.NumberOfEmployees + 1
27link );
28link
29link List<SObject> oldObjects = new List<SObject>{ acc, accTwo, accThree } ;
30link List<SObject> newObjects = new List<SObject>{ newAcc, accTwo, accThreeNew };
31link
32link ObjectChangeProcessor processor = new ObjectChangeProcessor(oldObjects, newObjects);
33link
34link List<Account> accounts = (List<Account>)
35link processor
36link .filterByChangeInField(Account.NumberOfEmployees)
37link .toList(new List<Account>());
38link
39link System.assertEquals(2, accounts.size());
40link System.assertEquals(7, accounts[0].NumberOfEmployees);
41link System.assertEquals(7, accounts[1].NumberOfEmployees);
42link }
43link}
Writing a test like this — documentation, in and of itself — is my preferred method for investigating a foreign object's API. Does it perform like I expect it to? Does it require complicated arguments to setup and maintain? The further you deviate from the SFDC included library for Apex, the harder it is going to be for somebody else to use.
When examining the original version of LazyFilterIterator
's "hasNext" implementation, Aidan suggested that it might not be stack-safe. Apex allows for a maximum stack depth of 1000 units, and running up against that boundary condition wouldn't be covered by the upcoming QueueableTimer
tests that you'll see below; because Apex Triggers artificially chunk operations into 200 record increments, it might lead to a false sense of confidence in the code's ability to process large amounts of objects. After tweaking the existing recursive function, I wrote the following test:
1link@isTest
2linkstatic void it_should_not_blow_the_stack_while_filtering() {
3link //that oughtta' do it!
4link Integer sentinelValue = 10^7;
5link List<Account> accounts = new List<Account>();
6link for(Integer index = 0; index < sentinelValue; index++) {
7link accounts.add(new Account(Name = 'Test ' + index));
8link }
9link
10link ObjectChangeProcessor processor = new ObjectChangeProcessor(accounts);
11link List<Object> sameAccounts = processor
12link .filter(new AlwaysTrue())
13link .toList(new List<Account>());
14link
15link System.assertEquals(sentinelValue, sameAccounts.size());
16link System.assert(true, 'Should make it here');
17link}
18link
19linkclass AlwaysTrue implements BooleanFunction {
20link public Boolean isTrueFor(Object o) { return true; }
21link}
And the test passed. Joy. As an aside, I typically don't advocate for testing implementation details (the iterator should work the same regardless of the number of records!); that said, on SFDC, it's always advisable to have bulkified tests to verify that you don't exceed your SOQL/SOSL/DML allowances, and assuring that your custom iterator isn't going to blow up on a large data-set certainly falls into this bulkified testing mandate.
Now I want to move on towards achieving feature parity through the LazyIterator
with the code shown earlier for the AccountHandler
object; namely, how can I load the iterator with functions that can act upon the SObjects passed into the trigger. This involves a sad case of boilerplate due to not being able to cast on a Iterator<SObject>
to Iterator<Object>
. Let's go back to the ObjectChangeProcessor
:
1linkpublic interface Function {
2link void call(Object o);
3link}
1linkpublic class ObjectChangeProcessor {
2link private LazyIterator iterator;
3link private List<Function> functions;
4link
5link public ObjectChangeProcessor(List<SObject> oldObjects, List<SObject> newObjects) {
6link this(new LazySObjectPairIterator(oldObjects, newObjects));
7link }
8link
9link/*alas, this constructor leads to the dreaded
10link"Operation cast is not allowed on type: System.ListIterator<SObject>" error
11linkpublic ObjectChangeProcessor(List<SObject> records) {
12link this((Iterator<Object>)records.iterator());
13link}*/
14link
15link public ObjectChangeProcessor(List<SObject> records) {
16link //so we have to do this instead :-\
17link this(new LazySObjectIterator(records.iterator()));
18link }
19link
20link private ObjectChangeProcessor(LazyIterator iterator) {
21link this.iterator = iterator;
22link this.functions = new List<Function>();
23link }
24link
25link public ObjectChangeProcessor addFunction(Function func) {
26link this.functions.add(func);
27link return this;
28link }
29link
30link public void process() {
31link this.iterator.forEach(this.functions);
32link }
33link}
And in the iterator inner class:
1linkpublic LazyIterator forEach(Function func) {
2link return this.forEach(new List<Function>{ func });
3link}
4linkpublic LazyIterator forEach(List<Function> funcs) {
5link while(this.hasNext()) {
6link//it's iterators all the way down!
7link Iterator<Function> funcIterator = funcs.iterator();
8link Object nextObject = this.next();
9link while(funcIterator.hasNext()) {
10link Function func = funcIterator.next();
11link func.call(nextObject);
12link }
13link }
14link return this;
15link}
Plus we need to add the LazySObjectIterator
inner class since casting on the Iterator
object is not allowed:
1linkvirtual class LazySObjectIterator extends LazyIterator {
2link private final Iterator<SObject> iterator;
3link public LazySObjectIterator(Iterator<SObject> iterator) {
4link super();
5link this.iterator = iterator;
6link }
7link
8link public override Boolean hasNext() {
9link return this.iterator.hasNext();
10link }
11link
12link public override Object next() {
13link return this.iterator.next();
14link }
15link}
Going back to our AccountHandler
example, it's time to encapsulate the phone/name update methods within classes:
1linkpublic class AccountHandler extends TriggerHandler {
2link
3link public override void beforeInsert(List<SObject> insertedRecords) {
4link new ObjectChangeProcessor(insertedRecords)
5link .addFunction(new NameNormalizer())
6link .addFunction(new PhoneNormalizer())
7link .process();
8link }
9link
10link class NameNormalizer implements Function {
11link public void call(Object o) {
12link Account acc = (Account)o;
13link acc.Name = acc.Name.normalizeSpace();
14link }
15link }
16link
17link class PhoneNormalizer implements Function {
18link public void call(Object o) {
19link Account acc = (Account)o;
20link acc.Phone = this.formatPhoneNumber(
21link //strip non-digits
22link acc.Phone.replaceAll(
23link '[^0-9]',
24link ''
25link )
26link );
27link }
28link
29link private String formatPhoneNumber(String phone) {
30link if(phone.length() == 10) {
31link return '(' + phone.substring(0, 3) + ') '
32link + phone.substring(3, 6) + '-'
33link + phone.substring(6);
34link } else if (phone.length() == 11
35link && phone.substring(0) == '1') {
36link return this.formatPhoneNumber(
37link phone.substring(
38link 1, phone.length() - 1)
39link );
40link }
41link return phone;
42link }
43link }
44link}
Note that testing the NameNormalizer
and PhoneNormalizer
inner classes is easily achievable, and they can also be broken out of the Handler into individual/wrapped classes as their responsibilities increase.
Now that the AccountHandler
code has been updated, it's finally time to re-run the QueueableTimer
object to see how lazy iteration stands up, performance-wise. Note, again, that I take the average of many runs when reporting out on performance. In other news, "finally time" turned out to be ~4 hours of writing between the "QueueableTimer" runs — whoah!
1link$Starting execute after gathering sample records, time passed: 0.363 seconds
2link$Ending, time passed: 32.416 seconds
Handler Method | Time | % Diff | Comments |
---|---|---|---|
Empty | 32.035s | 0.00% | Without any logic at all |
Standard for loops | 33.469s | 4.47% | Two calls to "for" loop iteration methods |
LazyIterator | 32.416s | 1.19% | Two "function" classes added to iterator |
Plus, the results of the tests in ObjectChangeProcessorTests
:
TEST NAME | OUTCOME | RUNTIME (MS) |
---|---|---|
it-should-call-functions-added-to-processor | Pass | 16 |
it-should-correctly-filter-records | Pass | 8 |
it-should-not-blow-the-stack-while-filtering | Pass | 5 |
I'll take 5ms to iterate 10 million rows, yes please.
In general, I would hasten to say two things regarding the performance of the LazyIterator
— both the vanilla for
loop and Lazy Iterator approach were tested dozens of times and an average of their results were taken. That said, the standard deviation for both approaches is large enough that I would caution taking the results too seriously. While I don't find it hard to believe that relying heavily on the native iterators outperforms the additional cost of initializing objects, neither do I find the performance gain in real terms to be the deciding factor in adopting this framework.
Reverse-engineering (an admittedly extremely small portion of) the LazyIterator
proved to be good, clean fun. Like all good exercises, it left me with plenty of inspiration for how to apply the code to my own use-cases. While I had a few nits with the overall level of verbosity, in general I would say that the framework code is both well-annotated with Javadoc descriptions and remarkably expressive at a very high-level of abstraction — no easy feat. I left impressed, which is my highest praise.
I will definitely be making use of some of the code here and from the Nebula Consulting repo. I like the fluent nature of working with the wrapped iterator; I can also see, with some work, how I would expand upon the structure orchestrated here to accommodate my two other most frequent use-cases:
Map<Id, Object>
or Map<String, Object>
that matches a value in that list, and performing processingMap<Id, List<Object>>
or Map<String, List<Object>>
that matches a value in that list, and performing processingAdditionally, there are some fun considerations for the LazyIterator
— some of which are handled within the existing Nebula Consulting LazyIterator
framework, some of which would be excellent additions:
Function
definition, but really you would be looking for a combination of the existing Function
and BooleanFunction
implementations, where all matches were tested for and, conditionally, processing was done if the result matched. Of course, depending on your business logic, there definitely exists the potential for independent updates to depend on one another in some happy temporal soup. Traditionally, showing that the order of functions being called matters makes use of explicit passing of variables to the further-down-the-line functions to make the coupling explicit. With a fluent iterator, another approach would be necessary; in looking again at the Nebula Repo, their ForkIterator
does handle the first use-case (independent filtering), but massaging the API to better broadcast dependent forking is only a dream at the momentEmptyIterator
object is forthcoming!LazyIterator
would be able to both build a one-to-one (Map<Id, Object>
or Map<String, Object>
) or one-to-many (Map<Id, List<Object>>
or Map<String, List<Object>>
) collection as part of a Function
and pass the results to future Function
s for usage. Unfortunately, while casting is a "pain" with Lists, it's not even allowed with Maps in Apex, which would probably necessitate painful (and limiting) serialization/deserialization techniques (which has actual performance implications, as well)As always, I hope that this post proved illuminating — if not on the seemingly endless iteration topic, then at least in having walked this road with me for some time. It's always appreciated.
Till next time!
The original version of Lightweight Trigger Handler can be read on my blog.
Home Apex Logging Service Apex Object-Oriented Basics Batchable And Queueable Apex Building A Better Singleton Continuous Integration With SFDX Dependency Injection & Factory Pattern Enum Apex Class Gotchas Extendable Apis Formula Date Issues Future Methods, Callouts & Callbacks Idiomatic Salesforce Apex Introduction & Testing Philosophy Lazy Iterators Lightweight Trigger Handler LWC Composable Modal LWC Composable Pagination LWC Custom Lead Path Mocking DML React Versus Lightning Web Components Refactoring Tips & Tricks Replacing DLRS With Custom Rollup Repository Pattern setTimeout & Implementing Delays Sorting And Performance In Apex Test Driven Development Example Testing Custom Permissions Writing Performant Apex Tests