Read more at jamessimone.net, or return to the homepage





Replacing DLRS With Custom Rollup

A few months ago I was tasked with replacing Declarative Lookup Rollup Summaries (DLRS) in an org suffering from frequent deadlocks. Rollup summary fields in Salesforce are plagued by severe limitations -- only being available on master-detail relationships being just the start of the list. When faced with implementing custom rollups, most people go with DLRS because it's free. Performance in the org inevitably suffers.

Read on to learn about how I built Rollup, complete with elastic scaling (go fast when you need to, slow when there are more rollups to process), to assist in orgs looking for DLRS-like flexibility with a much smaller performance overhead.

Discover how the "Google Analytics" approach to implementing rollups in your Salesforce org gives you the flexibility and power that you need - install with:

OR opt-into truly powerful code-based hooks that allow you to perform custom-based filtering using a simple Evaluator interface.

Keep reading, or check out the repository Readme for more!

Use Cases For Custom Rollups

Let's look at some of the problem areas experienced when implementing rollup fields:

The list could go on and on. None of these use cases are supported with out-of-the-box Salesforce. You can see in many areas of the platform that providing developers with ways to augment the existing behavior for out-of-the-box features leads to awesome customizations. With rollups, we don't have the ability through the UI to specify something like an overriding Apex class; we have no interface to implement. No -- if we want something better, we have to build it ourselves.

linkIntroducing Rollup

With the stage set and the problem well-defined, let's take a look at the beginnings of the Rollup project by examining how to invoke it from within a Trigger Handler class:

1link// within the "MyCustomObject__c" trigger handler

2link

3link// I'm aware it's not the sexiest interface. Keeping track of which field is which

4link// is now engraved in my head, but I'll annotate for now

5link// I expect most people will be using CMDT to manage these fields, anyway

6link

7linkRollupCalculator.sumFromTrigger( // the rollup operation

8link MyCustomObject__c.Amount__c, // the field that will inform the rollup

9link MyCustomObject__c.OwnerId, // the field on MyCustomObject related to the next argument, the matching field on the related object

10link User.Id, // the matching field on the object where the rollup will be performed

11link User.TotalCustomAmount__c, // the field where the rollup will be written to

12link new IsFlaggedBySalesUser(), // optional - a way to filter the "newCustomObjects" for ones that match

13link User.SObjectType // the related object where the rollup will be informed

14link);

15link

16linkprivate class IsFlaggedBySalesUser implements Rollup.Evaluator {

17link public Boolean matches(Object calcItem) {

18link MyCustomObject__c customObj = (MyCustomObject__c)calcItem;

19link return customObj.IsFlaggedBySalesUser__c;

20link }

21link}

In order to create something within Apex that can capture everything necessary for performing rollups in a generic way, many arguments are required. You could get rid of that last argument -- the SObjectType -- but, sadly, there is no method on the existing DescribeFieldResult class that points to the object it was initialized from.

linkGetting The SObjectType From An SObjectField

There is a workaround that's been floating around the Salesforce Stack Exchange for several years:

1linkpublic static SObjectType getSObjectType(Schema.SObjectField field) {

2link // This is a solution that was proposed on the Salesforce stack exchange

3link // and is the only work-around to a native lookup

4link // that I have been able to find.

5link Integer fieldHash = ((Object)field).hashCode();

6link

7link // Build a map of hashcodes for each fieldDescribe token

8link Map<String, Schema.SObjectType> globalDescribe = Schema.getGlobalDescribe();

9link Map<Integer,Schema.SObjectType> fieldHashCodeToSObjectTypeMap = new Map<Integer,Schema.SObjectType>();

10link for (String sobjname: globalDescribe.keySet()) {

11link for (Schema.SObjectField sobjField : globalDescribe.get(sObjName).getDescribe().fields.getMap().values())

12link fieldHashCodeToSObjectTypeMap.put(((Object) sObjField).hashCode(), globalDescribe.get(sobjName));

13link }

14link

15link // hard to believe it, but this actually works! it's a testament to the Describe objects, really:

16link // it means that any SObjectField token is a singleton! still, I would NEVER use this in production-level code

17link return fieldHashCodeToSObjectTypeMap.get(fieldHash);

18link}

Yikes. While that works, it falls down when you want to further generalize; it only works with strongly-typed SObjectFields; while the use of SObjectField types is a great experience for the developer, they too have shortcomings -- like not supporting parent or child relationships. While it's perfectly valid, using the MyCustomObject__c SObject from above, to reference MyCustomOject__c.Owner.Name, for example, you cannot construct an SObjectField to represent that relationship. Likewise with child relationships. The solution to replace DLRS will need strong ties into an easy-to-use invocable method, and sadly SObjectField is not yet supported as an argument type for Invocable Apex actions.

Additionally, a note on design -- after even the briefest of forays into the DLRS codebase had me clawing at my eyes, I made the decision to only use one class (and a test class). Isolating the mechanics of the rollups solely within the Rollup class would mean striking a delicate balance between creating a "god class" and adhering to the Single Responsibility Principle.

Shortcomings aside, the core code in Rollup is worth examining -- let's go deeper.

linkImplementing Rollups In Apex

Without further ado, let's look at some of the key methods in the initial version of Rollup:

1link// in Rollup

2link// AVG, MAX, MIN, COUNT operations not yet implemented

3link// because SUM was the only initial ask

4linkprivate enum Op {

5link SUM,

6link UPDATE_SUM,

7link DELETE_SUM

8link}

9link

10link// for really powerful up-front filtering of which items

11link// are rolled up, supplying an implementation of the Evaluator

12link// interface does the trick nicely

13linkpublic interface Evaluator {

14link Boolean matches(Object calcItem);

15link}

16link

17link// refer to the "IsFlaggedBySalesUser" example above

18linkprivate static List<SObject> filter(List<SObject> calcItems, Evaluator eval) {

19link List<SObject> applicableItems = new List<SObject>();

20link for(SObject calcItem : calcItems) {

21link if(eval != null && eval.matches(calcItem)) {

22link applicableItems.add(calcItem);

23link }

24link }

25link return applicableItems;

26link}

27link

28link// key the SObjects passed in to the String value

29link// matching the key on the object where the rollup

30link// will be performed

31linkprivate Map<String, List<SObject>> getCalcItemsByLookupField() {

32link Map<String, List<SObject>> lookupFieldToCalcItems = new Map<String, List<SObject>>();

33link for(SObject calcItem : this.calcItems) {

34link String key = (String)calcItem.get(this.lookupFieldOnCalcItem);

35link if(lookupFieldToCalcItems.containsKey(key) == false) {

36link lookupFieldToCalcItems.put(key, new List<SObject>{ calcItem };

37link } else {

38link lookupFieldToCalcItems.get(key).add(calcItem);

39link }

40link }

41link return lookupFieldToCalcItems;

42link}

43link

44link// your garden-variety dynamic SOQL

45linkprivate List<SObject> getLookupItems(Set<String> objIds) {

46link String queryString =

47link 'SELECT Id, ' + this.lookupObjOpField.getDescribe().getName() +

48link '\nFROM ' + this.lookupObj.getDescribe().getName() +

49link '\nWHERE ' + this.lookupField.getDescribe().getName() + ' = :objIds';

50link return Database.query(queryString);

51link}

52link

53link// the meat of the Rollup

54linkprivate void performRollup(Map<String, List<SObject>> calcItemsByLookupField, List<SObject> lookupItems) {

55link List<SObject> toUpdate = new List<SObject>();

56link for(SObject lookupRecord : lookupItems) {

57link String key = (String)lookupRecord.get(this.lookupField);

58link if(calcItemsByLookupField.containsKey(key) == false) {

59link continue;

60link }

61link

62link List<SObject> calcItems = calcItemsByLookupField.get(key);

63link Object priorVal = lookupRecord.get(this.lookupObjOpField);

64link Object newVal = this.getRollupVal(calcItems, priorVal);

65link lookupRecord.put(this.lookupObjOpField, newVal);

66link toUpdate.add(lookupRecord);

67link }

68link

69link update toUpdate;

70link}

71link

72link// right now, this "works" - but we'll need to further generalize

73link// to support other kinds of rollup operations

74linkprivate Object getRollupVal(List<SObject> calcItems, Object priorVal) {

75link Decimal returnVal = priorVal == null ? 0 : (Decimal)priorVal;

76link for(SObject calcItem : calcItems) {

77link switch on this.op {

78link when SUM {

79link returnVal += (Decimal)calcItem.get(this.opField);

80link }

81link when DELETE_SUM {

82link returnVal -= (Decimal)calcItem.get(this.opField);

83link }

84link when UPDATE_SUM {

85link Decimal oldVal = (Decimal)this.oldCalcItems.get(calcItem.Id).get(this.opField);

86link Decimal newVal = (Decimal)calcItem.get(this.opField);

87link returnVal += (newVal - oldVal); // could be negative, could be positive

88link }

89link when else {

90link throw new IllegalArgumentException('Other rollup op: ' + this.op.name() + ' not yet implemented');

91link }

92link }

93link }

94link return returnVal;

95link}

The framework for rolling values up appears fairly quickly; this isn't really even that many lines of code. Indeed, the rest of the Rollup is largely defined by constructors at this point, and static methods exposing the rollup operations.

linkReduce, Reuse, Refactor: Rollup, Part Two

First things first -- that getRollupVal method needs to decouple itself from the type of rollup being performed. This is a great use-case for inner classes:

1linkprivate Object getRollupVal(RollupCalculator calc, List<SObject> calcItems, Object priorVal) {

2link Rollup rollup = this.getRollupType(priorVal);

3link for (SObject calcItem : calcItems) {

4link rollup.performRollup(calc.op, priorVal, calcItem, calc.oldCalcItems, calc.opField.getDescribe().getName());

5link }

6link return rollup.getReturnValue();

7link}

8link

9linkprivate Rollup getRollupType(Object priorVal) {

10link // fun fact - integers, doubles, longs, and decimals

11link // will ALL return true here

12link if (priorVal instanceof Decimal) {

13link return new DecimalRollup(priorVal);

14link } else {

15link throw new IllegalArgumentException('Rollup operation not defined for: ' + JSON.serialize(priorVal));

16link }

17link}

18link

19linkprivate abstract class Rollup {

20link protected Object returnVal;

21link public Rollup(Object returnVal) {

22link this.returnVal = returnVal;

23link }

24link public Object getReturnValue() {

25link return returnVal;

26link }

27link public abstract void performRollup(Op op, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, String operationField);

28link}

29link

30linkprivate class DecimalRollup extends Rollup {

31link public DecimalRollup(Object priorVal) {

32link super(priorVal == null ? 0 : priorVal);

33link }

34link

35link public override void performRollup(Op operation, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, String operationField) {

36link Decimal returnVal = (Decimal) this.returnVal;

37link switch on operation {

38link when SUM {

39link returnVal += (Decimal) calcItem.get(operationField);

40link }

41link when DELETE_SUM {

42link returnVal -= (Decimal) calcItem.get(operationField);

43link }

44link when UPDATE_SUM {

45link Decimal oldVal = (Decimal) oldCalcItems.get(calcItem.Id).get(operationField);

46link Decimal newVal = (Decimal) calcItem.get(operationField);

47link returnVal += (newVal - oldVal); // could be negative, could be positive

48link }

49link when else {

50link throw new IllegalArgumentException('Other rollup op: ' + operation.name() + ' not yet implemented');

51link }

52link }

53link }

54link}

If you're looking at getRollupType in the above example and thinking that it looks like the Factory pattern, that's a bingo! Now the type of rollup being performed has been decoupled from the logic necessary to perform the actual rollup. That makes it easy to add in:

A potential code smell (perhaps obscured by only having one rollup operation defined) is the switch statement in DecimalRollup. This piece of logic will have to be replicated in each Rollup inner class prior to us being able to proceed. However, with a slight shift in perspective comes an object-oriented opportunity to reduce the boilerplate necessary to introduce new rollup types into the mix:

1link// in Rollup.cls

2linkprivate enum Op {

3link SUM,

4link UPDATE_SUM,

5link DELETE_SUM,

6link COUNT, // our first new operation!

7link UPDATE_COUNT,

8link DELETE_COUNT

9link}

10link

11linkprivate Rollup getRollupType(Object priorVal, Op operationType) {

12link // have to use the fully qualified Op name here (including the outer class)

13link // since its type is shadowed in this method

14link if(operationType.name().contains(RollupCalculator.Op.COUNT.name())) {

15link return new CountRollup(priorVal);

16link } else if (priorVal instanceof Decimal) {

17link return new DecimalRollup(priorVal);

18link } else {

19link throw new IllegalArgumentException('Rollup operation not defined for: ' + JSON.serialize(priorVal));

20link }

21link}

22link

23linkprivate abstract class Rollup {

24link protected Object returnVal;

25link public Rollup(Object returnVal) {

26link this.returnVal = returnVal;

27link }

28link // we make this virtual to deal with downcasting

29link public virtual Object getReturnValue() {

30link return returnVal;

31link }

32link public abstract void performRollup(Op op, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, SObjectField operationField);

33link}

34link

35linkprivate virtual class DecimalRollup extends Rollup {

36link public DecimalRollup(Object priorVal) {

37link // much as it pains me to duplicate the null check, it must be done;

38link // we can't reference instance methods till after the super() call

39link super(priorVal == null ? 0 : priorVal);

40link }

41link

42link protected Decimal getDecimalOrDefault(Object potentiallyUnitializedDecimal) {

43link return (Decimal) (potentiallyUnitializedDecimal == null ? 0 : potentiallyUnitializedDecimal);

44link }

45link

46link protected virtual Decimal getNumericValue(SObject calcItem, SObjectField operationField) {

47link return this.getDecimalOrDefault(calcItem.get(operationField));

48link }

49link

50link protected virtual Decimal getNumericChangedValue(SObject calcItem, SObjectfield operationField, Map<Id, SObject> oldCalcItems) {

51link Decimal newVal = this.getNumericValue(calcItem, operationField);

52link Decimal oldVal = this.getNumericValue(oldCalcItems.get(calcItem.Id), operationField);

53link // could be negative, could be positive ... could be 0!

54link return newVal - oldVal;

55link }

56link

57link public override void performRollup(Op operation, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, SObjectField operationField) {

58link Decimal returnVal = (Decimal) this.returnVal;

59link switch on operation {

60link when SUM, COUNT {

61link returnVal += this.getNumericValue(calcItem, operationField);

62link }

63link when DELETE_SUM, DELETE_COUNT {

64link returnVal -= this.getNumericValue(calcItem, operationField);

65link }

66link when UPDATE_SUM, UPDATE_COUNT {

67link returnVal += this.getNumericChangedValue(calcItem, operationField, oldCalcItems);

68link }

69link when else {

70link throw new IllegalArgumentException('Other rollup op: ' + operation.name() + ' not yet implemented');

71link }

72link }

73link }

74link}

75link

76linkprivate class CountRollup extends DecimalRollup {

77link public CountRollup(Object priorVal) {

78link super(priorVal);

79link }

80link

81link public override Object getReturnValue() {

82link return (Integer) this.returnVal;

83link }

84link

85link protected override Decimal getNumericValue(SObject calcItem, SObjectField operationField) {

86link Decimal potentialReturnValue = super.getNumericValue(calcItem, operationField);

87link return this.getCountValue(potentialReturnValue);

88link }

89link

90link protected override Decimal getNumericChangedValue(SObject calcItem, SObjectField operationField,

91link Map<Id, SObject> oldCalcItems) {

92link Decimal potentialReturnValue = super.getNumericChangedValue(calcItem, operationField, oldCalcItems);

93link return this.getCountValue(potentialReturnValue);

94link }

95link

96link private Decimal getCountValue(Decimal potentialReturnValue) {

97link return potentialReturnValue > 0 ? 1 : potentialReturnValue;

98link }

99link}

Et voila -- in just a few short lines and an additional check in the factory method, we've added a completely different rollup type into the mix. The new CountRollup class has an interesting condition to it (consistent with the rules for how the COUNT function works in SOQL) -- it still requires there to be a non-null value on the field within the calculation item that it's comparing to. It may be the case that people simply want to count all child/related objects and roll up their presence to a parent/related object. I'll be curious to hear from you as to whether or not that functionality is desired, prior to implementing a so-called BlindCount version.

As I added more functionality, it occurred to me that either by wanting to perform many rollups from within a single trigger, or by having rollup operations involving large amounts of records, it might be possible to creep towards the synchronous DML limit of 10,000 DML rows in a single transaction. Making Rollup perform the bulk of the work async also led to implementing the Data Processor pattern -- by burning a few SOQL queries on the synchronous side, Rollup can automatically (or through the use of RollupLimit__mdt custom metadata) scale from running as a Queueable to a Batchable as the size of rollup operations grow. This is the power of elastic scaling!


linkRollup - A Note On Progress

I wrote the above paragraph on December 21st, and jotted down a note to myself:
TODO:
max/min ??.

Today is December 29th. It's not that I took a break from writing this article; I didn't. What happened? Over the hours (and then days) that followed, I began to add additional operations for Rollup. My assumption -- with the SUM operations for numbers basically "complete" -- surrounding the ease of implementing the rest of the rollup functions slowly began to wither on the vine. I laughed on Christmas when a stranger submitted a PR on one of my open-source repos -- one thing I was no stranger to was putting in long hours on passion projects, and somebody else was clearly taking advantage of the holidays.

What began as a casual foray into adding COUNT quickly began to escalate outwards as the code smell that I mentioned earlier -- the switch statement in DecimalRollup became bigger and bigger.

"This isn't really even that many lines of code" - a younger, more naive version of me (earlier in this article), before the switch statement in DecimalRollup ended up as 50% of the size of the entire Rollup class when I started

The addition of an Invocable entry point for Flows and Process Builders contributed to the rising tide of lines of code -- and at the end of that journey, without many of the rollup functions even implemented yet, I realized it was past time ... the tests needed to be ported over, and new ones created ASAP to ensure what I had so far was going to work.

linkDesign Decisions For Testing The Rollup Framework

If you've read The Joys Of Apex before, you know that keeping your Salesforce tests running quickly is something I'm passionate about. This was doubly important in the current context: the tests I'd originally written were tightly coupled to two things that weren't going to make the cut:

One of the reasons that DLRS is so big as a codebase is because it imports a ton of code from FFLib to perform the metadata deployments necessary to create DLRS triggers/rollups as needed. It occurred to me that I was trying to encourage something more along the lines of adding an analytics tag to a website than the sort of service DLRS offerred: I've worked on several analytics implementations, and vetted many vendors. I've never heard of somebody offering a service that didn't require the installation of a JavaScript / backend SDK in order to work (even Cloudflare and other reverse proxy services require up-front configuration). In other words, as I considered how I wanted my tests to work, I had to also solidify the concepts (like Rollup being more akin to installing Google Analytics) necessary for Rollup to be used.

Since my prior tests were a no-go, I found myself recalling the immortal words of Kent Beck in "Test Driven Development By Example":

You will often be implementing TDD in code that doesn't have adequate tests. When you don't have enough tests, you are bound to come across refactorings that aren't supported by tests ... what do you do? Write the tests you wish you had.

In order to make the tests run fast, I needed to limit the amount of DML performed. Crucial to that effort would be the following class and seam:

1link// in Rollup.cls

2link@testVisible

3linkprivate virtual class DMLHelper {

4link public virtual void doUpdate(List<SObject> recordsToUpdate) {

5link update recordsToUpdate;

6link }

7link}

8link

9link/**

10link * receiving an interface/subclass from a property get/set (from the book "The Art Of Unit Testing") is an old technique;

11link * useful in limited contexts to get around the classic approach to dependency injection

12link * (such as in this case, when constructor-based DI isn't possible).

13link * It's more palatable in Apex than in many other languages, as a matter of fact -

14link * this is because the @testVisible annotation enforces for us the override only being possible while testing

15link */

16link@testVisible

17linkprivate static DMLHelper DML {

18link get {

19link if (DML == null) {

20link DML = new DMLHelper();

21link }

22link return DML;

23link }

24link set;

25link}

26link

27link// and then in the tests:

28link

29linkprivate class DMLMock extends Rollup.DMLHelper {

30link public List<SObject> Records = new List<SObject>();

31link public override void doUpdate(List<SObject> recordsToUpdate) {

32link this.Records = recordsToUpdate;

33link }

34link}

Wait a minute. Doesn't this look familiar? It should. But because I was intent on keeping everything within one class, I couldn't import dependencies -- I either needed to recreate them inside of the Rollup class, or do without them. The Factory pattern for dependency injection and the Repository pattern for strongly-typed and easily mocked queries were both discarded as a result. That left me with the DML Mock pattern -- and sure enough, all of the tests are lightning-fast as a result. Here's a simple one:

1link// in RollupTests.cls

2link@isTest

3linkstatic void shouldSumFromTriggerAfterInsert() {

4link DMLMock mock = getMock(new List<Opportunity>{ new Opportunity(Amount = 25), new Opportunity(Amount = 25) });

5link Rollup.triggerContext = TriggerOperation.AFTER_INSERT;

6link

7link Rollup rollup = Rollup.sumFromTrigger(

8link Opportunity.Amount,

9link Opportunity.AccountId,

10link Account.Id,

11link Account.AnnualRevenue,

12link Account.SObjectType

13link );

14link

15link System.assertEquals(true, mock.Records.isEmpty());

16link

17link Test.startTest();

18link rollup.runCalc();

19link Test.stopTest();

20link

21link System.assertEquals(1, mock.Records.size(), 'Records should have been populated SUM AFTER_INSERT');

22link Account updatedAcc = (Account) mock.Records[0];

23link System.assertEquals(50, updatedAcc.AnnualRevenue, 'SUM AFTER_INSERT should add the original opportunity amount');

24link}

One hidden side-effect is the tying of the Opportunities to the Account in question - that happens in the getMock method:

1link// in RollupTests.cls

2linkprivate static DMLMock getMock(List<SObject> records) {

3link Account acc = [SELECT Id FROM Account];

4link for (SObject record : records) {

5link record.put('AccountId', acc.Id);

6link }

7link

8link return loadMock(records);

9link}

10link

11link// ...

12link

13linkprivate static DMLMock loadMock(List<SObject> records) {

14link Rollup.records = records;

15link Rollup.shouldRun = true;

16link DMLMock mock = new DMLMock();

17link Rollup.DML = mock;

18link

19link return mock;

20link}

There are, in fact, only three other helper methods (non test methods) in the entire test class. Once I had tests that covered the basics of what I'd written for the SUM and COUNT implementations, it was time to approach this thing TDD-style:

This simple rhythm helped me to immediately spot a flaw in the code for the Invocable method as I worked to create tests for this new functionality -- it was being fed into a method that used the current Trigger context (the TriggerOperation enum) to figure out what kind of rollup operation was being performed. That's also where the Rollup.records variable came from in the loadMock code, above; a way to stub in the trigger records without actually having to require a trigger being run.

When you look at the first ~20 lines of Rollup, you can see the use of these @testVisible private static variables as the "poor man's dependency injection:"

1link /**

2link * Test override / bookkeeping section. Normally I would do this through dependency injection,

3link * but this keeps things much simpler

4link */

5link @testVisible

6link private static Boolean shouldRun;

7link @testVisible

8link private static Boolean shouldRunAsBatch = false;

9link @testVisible

10link private static TriggerOperation triggerContext = Trigger.operationType;

11link @testVisible

12link private static Map<Id, SObject> oldRecordsMap;

13link @testVisible

14link private static List<Rollup__mdt> rollupMetadata;

15link @testVisible

16link private static List<SObject> queryRecords;

17link @testVisible

18link private static RollupLimit__mdt defaultRollupLimit;

19link @testVisible

20link private static RollupLimit__mdt specificRollupLimit;

There are some juicy hints, above, of what was ultimately to come.

linkAdding Custom Metadata-driven Rollups

Adding the CMDT-record driven rollups was simple now that I had a burgeoning test suite and two different possible points of entry (Invocable / Trigger-based) into Rollup. Indeed, because the test suite was expansive and I believe that CMDT rollups form the core of peoples' rollup needs, refactoring the code to support custom metadata as a first-class citizen became (truly) a joy. You can tell that the code is really built around it because there are only 3 lines of code in the public-facing method:

1link// in Rollup.cls - don't mind that null argument below, it's for the custom Evaluator interface

2linkpublic static void runFromTrigger() {

3link SObjectType sObjectType = getTriggerRecords().getSObjectType();

4link List<Rollup__mdt> rollupMetadata = getTriggerRollupMetadata(sObjectType);

5link runFromTrigger(rollupMetadata, null).runCalc();

6link}


This is the "Google Analytics" approach: in order to use Rollup, all you need to do is add one line of code to your triggers:

1linkRollup.runFromTrigger();

Unless you need a ton of customization, it's really as simple as that. Please note: this requires your trigger to use the following contexts: after insert, after update, and before delete. Without those in place, Rollup will not function as designed for trigger-based rollups!

The rest of the info -- about which rollups need to be processed for the trigger in question -- can all live in the Rollup__mdt Custom Metadata:

There are some peculiarities within Apex when working with Entity Definition and Field Definition-based Custom Metadata fields, which I will detail just below in the key takeaways section!

linkInvoking Rollup.cls From A Process Builder / Flow

Invoking the `Rollup` process from a Flow, in particular, is a joy; with a Record Triggered Flow, you can do the up-front processing to take in only the records you need, and then dispatch the rollup operation to the `Rollup` invocable:

Example flow

This is also the preferred method for scheduling; while I do expose the option to schedule a rollup from Apex, I find the ease of use in creating Scheduled Flows in conjunction with the deep power of properly configured Invocables to be much more scalable than the "Scheduled Jobs" of old. This also gives you the chance to do some truly crazy rollups -- be it from a Scheduled Flow, an Autolaunched Flow, or a Platform Event-Triggered Flow. As long as you can manipulate data to correspond to the shape of an existing SObject's fields, they don't even have to exist; you could have an Autolaunched flow rolling up records when invoked from a REST API so long as the data you're consuming contains a String/Id matching something on the "parent" rollup object.

linkKey Takeaways In Replacing DLRS

There were quite a few learning moments as I worked through rollup edge cases; I've chosen to spend the rest of this article articulating ones that I think might be helpful or interesting to you in your own Salesforce journey:

linkEntity Definition & Field Definition Custom Metadata Relationships Can Be Tricky

There's not a whole lot of documentation out there about Entity / Field Definition-basd CMDT. They are as good as they sound -- giving users of your CMDT object-level and field-level safety when letting them select fields, but something interesting that I found while working with Field Definition fields in Apex is that they are stored as "ObjectName.FieldName" in the database. This roughly corresponds to a string-level representation of what an SObjectField type is written as:

1link// if we query for the Rollup__mdt shown earlier

2linkRollup__mdt rollupMetadata = [SELECT RollupFieldOnLookupObject__c FROM Rollup__mdt LIMIT 1];

3linkSystem.debug(rollupMetadata.RollupFieldOnLookupObject__c); // outputs "Opportunity.Amount", for example

4linkSObjectField opportunityAmount = Opportunity.Amount;

5linkSystem.debug(opportunityAmount); // ouputs ... "Amount" ... so you know that somebody overrode the "toString()" method for this class!

When working with dynamic fields in SOQL, developers frequently use the DescribeSObjectResult and DescribeFieldResult classes that give you access to metadata about objects/fields ... but in this case, I found I had to create a helper method specifically for working with the String-based version of the Field Definition values coming in from the Rollup__mdt records:

1link// takes a string from CMDT like "Opportunity.Amount" and returns just the field name: "Amount"

2link// this allows us to match the String-based version of the field with its corresponding SObjectField

3link// by calling describeForSObject.fields.getMap().get(theFieldNameReturnedFromgetParedFieldName)

4linkprivate static String getParedFieldName(String fullFieldName, DescribeSObjectResult describeForSObject) {

5link return String.isBlank(fullFieldName) ? '' : fullFieldName.replace(describeForSObject.getName() + '.', '');

6link}

linkUsing Enums Is Great, But Instantiating Them From Strings Isn't Obvious

You might remember from the Apex Enum Class Gotchas article that sending the name() value for an enum is the only way to properly deserialize the enum when you're ingesting data either in Apex or in another service. That's all well and good -- but, as it turns out, you can't deserialize directly to the String-based enum:

1link// in Rollup.cls - making this a public enum doesn't change the result

2linkprivate enum Op {

3link SUM,

4link UPDATE_SUM,

5link DELETE_SUM

6link // etc

7link}

8link

9linkRollup__mdt rollupMetadata = methodWhereWeGetTheMetadata();

10link// this is a crazy thing and I wouldn't have wanted to do it anyway, but science ...

11link// if you ACTUALLY need to do these things from within Apex, I highly recommend the use of the JSONGenerator class

12linkString operationFromMetadata = '{ "op" : "' + rollupMetadata.RollupType__c + '"}';

13linkOp operation = (Op)JSON.deserialize(operationFromMetadata, Op.class);

14linkSystem.debug(operation);

15link// outputs null

16link

17link// you COULD make a wrapper class

18linkprivate class OpWrapper {

19link public Op op { get; set; }

20link}

21linkOpWrapper opWrapper = (OpWrapper)JSON.deserialize(operationFromMetadata, OpWrapper.class);

22linkSystem.debug(opWrapper);

23link// outputs: "OpWrapper:[Op=SUM]", for example

Keying the string from the CMDT to be valid JSON and creating a wrapper class left me feeling a little ill, so instead I went with a lazily-loaded Map<String, Op> using the included values() method present on all enums:

1linkprivate static Map<String, Op> opNameToOp {

2link get {

3link if (opNameToOp == null) {

4link opNameToOp = new Map<String, Op>();

5link for (Op operation : Op.values()) {

6link opNameToOp.put(operation.name(), operation);

7link }

8link }

9link return opNameToOp;

10link }

11link set;

12link}

It's not perfect, but it's better than crafting JSON to get around not being able to cast from a String to an enum, and not being able to reference the enum by its name() in any other way.

linkNot all fields of the same type in Salesforce support MIN or MAX operations

This is probably true of other field types (honestly, at this point, nothing would really surprise me), but it came up frequently for me while testing with Date / Datetime fields, as well as multi-select picklists (the Devil's picklists, some would say).

It did come as a real surprise to me as I was writing tests; I had used only the Account and Opportunity objects so far in my tests, in an effort to keep them as generic as possible. I wanted to write a test using a Date field that wasn't required on insert to test my DefaultFieldInitializer (more on that in a second). While you can MIN or MAX the Opportunity's CloseDate field, the "parent" object I had been using as the target of my rollups, Account, didn't have a Date field on it. In retrospect, I could have used the Contract object (and later I would), but I'm glad I didn't, as I might not have encountered this charming error message otherwise: System.QueryException: There's a problem with your query: field ActivityDate does not support aggregate operator MAX. Uhhh, OK. Thanks for that, Salesforce.

Wherever possible, I've tried to make the code resilient to the idiosyncracies of the platform; for operations like this, if that means doing the damn MIN / MAX myself, so be it:

1link// MIN/MAX is allowed, but not for all fields, and not consistently. Go figure!

2linkprotected virtual override Object calculateNewAggregateValue(Set<Id> excludedItems, Op operation, SObjectField operationField, SObjectType sObjectType) {

3link Object aggregate;

4link try {

5link aggregate = super.calculateNewAggregateValue(excludedItems, operation, operationField, sObjectType);

6link } catch (Exception ex) {

7link // technically a System.QueryException, but I figure we might as well catch em all and try like hell to aggregate anyway

8link Decimal minOrMax;

9link List<SObject> allOtherItems = Database.query('SELECT ' + operationField + ' FROM ' + sObjectType + ' WHERE Id != :excludedItems');

10link for (SObject otherItem : allOtherItems) {

11link Decimal otherItemDate = this.getDecimalOrDefault(otherItem.get(operationField));

12link if (otherItemDate != null && operation.name().contains(Op.MAX.name()) && (minOrMax == null || otherItemDate > minOrMax)) {

13link minOrMax = otherItemDate;

14link } else if (otherItemDate != null && operation.name().contains(Op.MIN.name()) && (minOrMax == null || otherItemDate < minOrMax)) {

15link minOrMax = otherItemDate;

16link }

17link }

18link if (minOrMax == null) {

19link aggregate = operation.name().contains(Op.MIN.name()) ? FieldInitializer.maximumLongValue : FieldInitializer.minimumLongValue;

20link } else {

21link aggregate = minOrMax;

22link }

23link }

24link

25link return aggregate;

26link}

27link// not pictured - the "hot as hell" section where min/max gets tabulated for multi-select picklists. Yowza! Feel the burn!

linkNull as a value doesn't retain its type information

I thought I'd be able to simply query a given field on the rollup object in question, test for its type using instanceof, and then move on to rolling up. Unfortunately, that approach failed on my very first test where the rollup object didn't have the rollup field initialized. I needed to find something -- anything -- that would give me a clue about what the type was for a given field at runtime, and found it in the DisplayType enum. Indeed, I found out after creating the default value initializer that this was the same method employed by a number of dynamic Apex test libraries:

1link// in Rollup.cls

2linkprivate virtual class DefaultSObjectFieldInitializer {

3link public final Datetime defaultDateTime = Datetime.newInstanceGmt(1970, 1, 1);

4link public final Long maximumLongValue = (Math.pow(2, 63) - 1).longValue();

5link public final Long minimumLongValue = this.maximumLongValue * -1;

6link

7link public virtual Object getDefaultValue(SObjectField field) {

8link DescribeFieldResult fieldDescribe = field.getDescribe();

9link if (fieldDescribe.isDefaultedOnCreate()) {

10link return fieldDescribe.getDefaultValue();

11link }

12link // not surprisingly, "getDefaultValue" on the DescribeFieldResult returns null for fields without default values

13link // this is a shame - all types *should* have default values. Instead, we have the privilege of getting to initialize them

14link Object initializedDefault;

15link switch on fieldDescribe.getType() {

16link when CURRENCY, DOUBLE, INTEGER, LONG, PERCENT {

17link initializedDefault = 0;

18link }

19link when DATETIME {

20link initializedDefault = this.defaultDateTime;

21link }

22link when DATE {

23link initializedDefault = this.defaultDateTime.dateGmt();

24link }

25link when TIME {

26link initializedDefault = this.defaultDateTime.timeGmt();

27link }

28link when STRING, ID, TEXTAREA, URL, PHONE, EMAIL{

29link initializedDefault = '';

30link }

31link when PICKLIST, MULTIPICKLIST {

32link // more on this part in a second

33link initializedDefault = new PicklistController(field.getDescibe()).getDefaultValue(field);

34link }

35link when else {

36link throw new IllegalArgumentException('Field: ' + field + ' of type: ' + fieldType.name() + ' specified invalid for rollup operation');

37link }

38link }

39link return initializedDefault;

40link }

41link}

linkFinding default SObject fields for different DisplayTypes is fun

In the end, this Anonymous Apex script proved invaluable for hunting down standardly available fields of special types like TIME:

1linkpublic static void printDisplayTypeInfo(DisplayType desiredType) {

2link Map<String, SObjectType> namesToTypes = Schema.getGlobalDescribe();

3link for(SObjectType sType : namesToTypes.values()) {

4link Map<String, SObjectField> fields = sType.getDescribe().fields.getMap();

5link for(String fieldName : fields.keyset()){

6link SObjectField field = fields.get(fieldName);

7link DescribeFieldResult describeResult = field.getDescribe();

8link if(describeResult.getType() == desiredType && describeResult.isUpdateable()) {

9link System.debug('SObjectType: ' + sType);

10link System.debug(describeResult.getName());

11link // you could put a return statement here to only print the first result found

12link }

13link }

14link }

15link}

I had never worked with the ContactPointAddress or ContactPointEmail objects before; they're relatively new additions to the system, and it was fun to learn a bit more about them while using them to wire up different zany relationships.

linkObject-Oriented Programming Is Extremely Powerful (All Dates Are Numbers)

I've talked quite a bit about the infamous switch statement now in the DecimalRollupCalculator. After implementing all of the rollup operations for numbers, I teetered at the precipice -- how to take the logic in DecimalRollupCalculator and generalize it so that other subclasses could make use of it. As I considered the horrors of a switch statement that needed to differentiate not only between different rollup operations, but also their context (was it an insert? an update? a delete??), my stomach began to twist. I had just written my first failing test for implementing MAX for Datetimes. Would my response really be to copy pasta?

Then it hit me. Datetimes are all stored uniformly within Salesforce in UTC time. UTC can be represented by numbers. Dates are just Datetimes with a zero'd out Time section. The game was on. In the end, this section truly took very little additional code to get right:

1link// omitting the Datetime parent class, which itself descends from DecimalRollupCalculator

2link// the reason should be clear if you take a peek at the source code. While it's simple, an

3link// excerpt from DatetimeRollupCalculator is shown below in the "SOQL Drops Milliseconds From Datetimes ..." section

4linkprivate class DateRollupCalculator extends DatetimeRollupCalculator {

5link // for Date, it's not necessary to override the "getDecimalOrDefault" method in DatetimeRollupCalculator

6link // because the conversion only happens in "getReturnValue"

7link public DateRollupCalculator(Object priorVal, SObjectField operationField) {

8link super(Datetime.newInstanceGmt((Date) priorVal, Time.newInstance(0, 0, 0, 0)), operationField);

9link }

10link

11link public override Object getReturnValue() {

12link return ((Datetime) super.getReturnValue()).dateGmt();

13link }

14link }

15link

16link private class TimeRollupCalculator extends DatetimeRollupCalculator {

17link public TimeRollupCalculator(Object priorVal, SObjectField operationField) {

18link super(Datetime.newInstanceGmt(FieldInitializer.defaultDateTime.dateGmt(), (Time) priorVal), operationField);

19link }

20link

21link public override Object getReturnValue() {

22link return ((Datetime) super.getReturnValue()).timeGmt();

23link }

24link

25link protected override Decimal getDecimalOrDefault(Object potentiallyUnitializedDecimal) {

26link Datetime defaultDatetime;

27link if (potentiallyUnitializedDecimal instanceof Time) {

28link defaultDatetime = Datetime.newInstanceGmt(FieldInitializer.defaultDateTime.dateGmt(), (Time) potentiallyUnitializedDecimal);

29link } else if (potentiallyUnitializedDecimal instanceof Decimal) {

30link defaultDatetime = Datetime.newInstance(((Decimal) potentiallyUnitializedDecimal).longValue());

31link } else {

32link defaultDatetime = FieldInitializer.defaultDateTime;

33link }

34link return defaultDatetime.getTime();

35link }

36link }

That commit -- and the powerful realization that I'd been able to add a possible sixty-three different permutations for rollup operations in what amounted to a mere ~32 lines of code ... that's the power of Object-Oriented Programming!

Edit -- what I failed to mention, upon originally publishing this article, was that there was a duplicated switch statement (for String-based rollups); a detail that had bothered me in the days leading up to release, but which I didn't have the time (or energy, after several marathon days spent "finishing" Rollup) to address prior to launching. Several days after releasing, I went back and made use of the Chain Of Responsibility to break up this duplication. As is common when virtual methods are added at the parent level, this actually ended up increasing the lines of code in the final Rollup file. At the same time, indentation (one of the great sins present in the use of switch statements) was much reduced. What was left? One switch statement to rule them all, with virtual methods to bind them. While lines of code can be a metric for complexity, it also often ends up being deceptive -- the code reads more like a story, now, where rollup operations are explicitly opted into, instead of having to scan through a switch statement to determine which operation leads to which result.

linkintValue() on a Long can be a wild ride

While working on MAX/MIN based code, I realized quickly that there were only two reliable sentinel values when it came to numbers -- the maximum possible number that would fit into a 64-bit number (a Long), and the minimum possible number. What bit me really, really hard while working on implementing min/max for picklists? To anwer that, first an aside -- min/maxing on a picklist is supposed to return the deepest possible entry in the picklist (for MAX), or the closest to the top of the picklist (for MIN). Picklists have many ... interesting ... subleties in Salesforce, and the implicit concept of "rank" (just look at the Path components for Lead Status or Opportunity Stage Name) is just one of their many quirks.

If a value doesn't exist on the lookup object, that value should always lose; on a MIN it should be greater than the "rank" of any other field (so that any comparison to it leads to a truthy "less than" evaluation); on a MAX it should be less than the rank of any other field (likewise; it should lead to a truthy "greater than" evaluation). Making an object to perform these evaluations for picklists was a great exercise in Object Oriented Programming -- frequently throughout this project, that proved to be the case. Making the PicklistController inner class descend from the DefaultFieldInitializer made sense in the context of the object hierarchy. Though the PicklistController had many more responsibilities, it could also handle setting the default value for a picklist with ease.

Where things finally took a turn for the worse was when my first picklist test for ensuring MIN/MAX was working correctly failed. The logic looked perfect -- what could be the issue? It took me a painful 30 minutes to see the issue, so deeply ingrained were my assumptions about the way that Long values would translate to Integers. Only at the last second did I truly comprehend that I had betrayed myself:

1linkprivate class PicklistController extends DefaultSObjectFieldInitializer {

2link// etc ...

3link

4link private Integer getSentinelValue(Boolean isMin) {

5link return (isMin ? this.maximumLongValue : this.minimumLongValue).intValue();

6link }

7link}

Passing Boolean values as method arguments is always to be strongly discouraged, but this method call was already on the tail end of a ternary and I felt, when creating it, that I had no other option. I will say, though, that I once had a coworker who was deeply passionate about using nested ternaries. I lost track of him over the years, but ... perhaps the issue would have been clearer to me if the culprit had been in a truly doubled ternary; after all, who in their right minds could avoid investigating such a travesty?

In any case. Do you know what the integer values are for the minimum and maximum Long values (-2^63 and 2^63, respectively)?

1linkLong maximumLongValue = (Math.pow(2, 63) - 1).longValue();

2linkLong minimumLongValue = maximumLongValue * -1;

3link

4linkSystem.debug(maximumLongValue); // prints: 9223372036854775807

5linkSystem.debug(minimumLongValue); // prints: -9223372036854775807

6link

7link// Now as ints!

8linkSystem.debug(maximumLongValue.intValue()); // prints -1

9linkSystem.debug(minimumLongValue.intValue()); // prints 1

Ouch. Well, that explained my failing test! My understanding (after a brief foray into the details) is that casting to Integer or calling intValue() is safe within the bounds of the 32-bit allowed integer sizes, but after that all bets are off. Creating the min/max integer bounds did the trick wonderfully. Running into edge cases like this can hurt, but it also expands the mind -- you only get bit by something like this once before knowing to look out for it next time around. Plus, I ended up being able to get rid of the Boolean passing altogether -- two birds with one stone!

linkSOQL drops the milliseconds from Datetime fields when they are retrieved from the database

This was a fun one -- and one that I was already aware of from lurking on the SFXD Discord. This is a real pain -- especially in testing --, but in the end the following bit did the trick nicely:

1link// one of the worst things about SOQL is that Datetimes retrieved have the millisecond values truncated

2linkDatetime datetimeWithMs = potentiallyUnitializedDecimal instanceof Decimal

3link ? Datetime.newInstance(((Decimal) potentiallyUnitializedDecimal).longValue())

4link : ((Datetime) potentiallyUnitializedDecimal);

5link// reading through the source code provides a more cogent rationale

6link// for the above eyesore over anything I can muster here.

7linkreturn Datetime.newInstanceGmt(

8link datetimeWithMs.yearGmt(),

9link datetimeWithMs.monthGmt(),

10link datetimeWithMs.dayGmt(),

11link datetimeWithMs.hourGmt(),

12link datetimeWithMs.minuteGmt(),

13link datetimeWithMs.secondGmt()

14link )


linkCustom Rollup Wrap-up

Well, it's out there now. This article -- and the corresponding code -- has consumed an enormous quantity of time since work began on it in earnest in early December. I would highly recommend developers check out the source code (and the travelogue style commit history). Over the coming months I plan to add more functionality to Rollup -- for now, I'm hopeful that you'll consider trying it out. It's efficient, scales elastically, allows for rollups on fields (like Task.ActivityDate) that don't always allow for rollups in SOQL, and is well-tested.

I'm aware that Rollup doesn't hit 100% feature-parity versus DLRS ... and though I have plans to meet that challenge, as well, I believe that we're well past the fabled "80% of the functionality" stage. If your org (like many out there) is struggling under the weight of DLRS' auto-spawned triggers, I'm confident that Rollup will be a valuable tool for both the declaratively-minded as well as the developers out there.


linkPostscript

My original intent was to finish this article by December 27th, the one year anniversary of the Joys Of Apex. Despite some insanely long days spent writing, that didn't quite happen. Despite that, I just wanted to say that the readers of this series have helped ease the burden in a year that was extremely challenging for many people. Stuck inside for large portions of time, I took to writing -- and it shows. More than ten of the articles I wrote over the last year came out during the first 2 months of the pandemic -- some only days apart from one another. I do not advertise, and have not attempted to monetise in any way the incredible surge of traffic my personal website has experienced as a result. My intent is to provide readers with free content and materials to refer back to. Indeed, the only work I've done on the site over the past year was immediately preceeding this article, as I spruced up the Joys Of Apex blog page to better show off the posts.

All of that is to say -- thank you for an incredible year. Here's to hoping that 2021 will prove a better year for the world, and for you.

The original version of Replacing DLRS With Custom Rollup can be read on my blog.

Use Cases For Custom RollupsIntroducing RollupGetting The SObjectType From An SObjectFieldImplementing Rollups In ApexReduce, Reuse, Refactor: Rollup, Part TwoRollup - A Note On ProgressDesign Decisions For Testing The Rollup FrameworkAdding Custom Metadata-driven RollupsInvoking Rollup.cls From A Process Builder / FlowKey Takeaways In Replacing DLRSEntity Definition & Field Definition Custom Metadata Relationships Can Be TrickyUsing Enums Is Great, But Instantiating Them From Strings Isn't ObviousNot all fields of the same type in Salesforce support MIN or MAX operationsNull as a value doesn't retain its type informationFinding default SObject fields for different DisplayTypes is funObject-Oriented Programming Is Extremely Powerful (All Dates Are Numbers)intValue() on a Long can be a wild rideSOQL drops the milliseconds from Datetime fields when they are retrieved from the databaseCustom Rollup Wrap-upPostscript

Home Advanced Logging Using Nebula Logger Apex Logging Service Apex Object-Oriented Basics Batchable And Queueable Apex Building A Better Singleton Continuous Integration With SFDX Dependency Injection & Factory Pattern Enum Apex Class Gotchas Extendable Apis Formula Date Issues Future Methods, Callouts & Callbacks Idiomatic Salesforce Apex Introduction & Testing Philosophy Lazy Iterators Lightweight Trigger Handler LWC Composable Modal LWC Composable Pagination LWC Custom Lead Path Mocking DML React Versus Lightning Web Components Refactoring Tips & Tricks Replacing DLRS With Custom Rollup Repository Pattern setTimeout & Implementing Delays Sorting And Performance In Apex Test Driven Development Example Testing Custom Permissions The Tao Of Apex Writing Performant Apex Tests



Read more tech articles