Read more at jamessimone.net, or return to the homepage





Replacing DLRS With Custom Rollup

A few months ago I was tasked with replacing Declarative Lookup Rollup Summaries (DLRS) in an org suffering from frequent deadlocks. Rollup summary fields in Salesforce are plagued by severe limitations -- only being available on master-detail relationships being just the start of the list. When faced with implementing custom rollups, most people go with DLRS because it's free. Performance in the org inevitably suffers.

Read on to learn about how I built Rollup, complete with elastic scaling (go fast when you need to, slow when there are more rollups to process), to assist in orgs looking for DLRS-like flexibility with a much smaller performance overhead.

Discover how the "Google Analytics" approach to implementing rollups in your Salesforce org gives you the flexibility and power that you need - install with:

OR opt-into truly powerful code-based hooks that allow you to perform custom-based filtering using a simple Evaluator interface.

Keep reading, or check out the repository Readme for more!

Use Cases For Custom Rollups

Let's look at some of the problem areas experienced when implementing rollup fields:

The list could go on and on. None of these use cases are supported with out-of-the-box Salesforce. You can see in many areas of the platform that providing developers with ways to augment the existing behavior for out-of-the-box features leads to awesome customizations. With rollups, we don't have the ability through the UI to specify something like an overriding Apex class; we have no interface to implement. No -- if we want something better, we have to build it ourselves.

linkIntroducing Rollup

With the stage set and the problem well-defined, let's take a look at the beginnings of the Rollup project by examining how to invoke it from within a Trigger Handler class:

1link// within the "MyCustomObject__c" trigger handler

2link

3link// I'm aware it's not the sexiest interface. Keeping track of which field is which

4link// is now engraved in my head, but I'll annotate for now

5link// I expect most people will be using CMDT to manage these fields, anyway

6link

7linkRollupCalculator.sumFromTrigger( // the rollup operation

8link  MyCustomObject__c.Amount__c, // the field that will inform the rollup

9link  MyCustomObject__c.OwnerId, // the field on MyCustomObject related to the next argument, the matching field on the related object

10link  User.Id, // the matching field on the object where the rollup will be performed

11link  User.TotalCustomAmount__c, // the field where the rollup will be written to

12link  new IsFlaggedBySalesUser(), // optional - a way to filter the "newCustomObjects" for ones that match

13link  User.SObjectType // the related object where the rollup will be informed

14link);

15link

16linkprivate class IsFlaggedBySalesUser implements Rollup.Evaluator {

17link  public Boolean matches(Object calcItem) {

18link    MyCustomObject__c customObj = (MyCustomObject__c)calcItem;

19link    return customObj.IsFlaggedBySalesUser__c;

20link  }

21link}

In order to create something within Apex that can capture everything necessary for performing rollups in a generic way, many arguments are required. You could get rid of that last argument -- the SObjectType -- but, sadly, there is no method on the existing DescribeFieldResult class that points to the object it was initialized from.

linkGetting The SObjectType From An SObjectField

There is a workaround that's been floating around the Salesforce Stack Exchange for several years:

1linkpublic static SObjectType getSObjectType(Schema.SObjectField field) {

2link  // This is a solution that was proposed on the Salesforce stack exchange

3link  // and is the only work-around to a native lookup

4link  // that I have been able to find.

5link  Integer fieldHash = ((Object)field).hashCode();

6link

7link  // Build a map of hashcodes for each fieldDescribe token

8link  Map<String, Schema.SObjectType> globalDescribe = Schema.getGlobalDescribe();

9link  Map<Integer,Schema.SObjectType> fieldHashCodeToSObjectTypeMap = new Map<Integer,Schema.SObjectType>();

10link  for (String sobjname: globalDescribe.keySet()) {

11link    for (Schema.SObjectField sobjField : globalDescribe.get(sObjName).getDescribe().fields.getMap().values())

12link      fieldHashCodeToSObjectTypeMap.put(((Object) sObjField).hashCode(), globalDescribe.get(sobjName));

13link  }

14link

15link  // hard to believe it, but this actually works! it's a testament to the Describe objects, really:

16link  // it means that any SObjectField token is a singleton! still, I would NEVER use this in production-level code

17link  return fieldHashCodeToSObjectTypeMap.get(fieldHash);

18link}

Yikes. While that works, it falls down when you want to further generalize; it only works with strongly-typed SObjectFields; while the use of SObjectField types is a great experience for the developer, they too have shortcomings -- like not supporting parent or child relationships. While it's perfectly valid, using the MyCustomObject__c SObject from above, to reference MyCustomOject__c.Owner.Name, for example, you cannot construct an SObjectField to represent that relationship. Likewise with child relationships. The solution to replace DLRS will need strong ties into an easy-to-use invocable method, and sadly SObjectField is not yet supported as an argument type for Invocable Apex actions.

Additionally, a note on design -- after even the briefest of forays into the DLRS codebase had me clawing at my eyes, I made the decision to only use one class (and a test class). Isolating the mechanics of the rollups solely within the Rollup class would mean striking a delicate balance between creating a "god class" and adhering to the Single Responsibility Principle.

Shortcomings aside, the core code in Rollup is worth examining -- let's go deeper.

linkImplementing Rollups In Apex

Without further ado, let's look at some of the key methods in the initial version of Rollup:

1link// in Rollup

2link// AVG, MAX, MIN, COUNT operations not yet implemented

3link// because SUM was the only initial ask

4linkprivate enum Op {

5link  SUM,

6link  UPDATE_SUM,

7link  DELETE_SUM

8link}

9link

10link// for really powerful up-front filtering of which items

11link// are rolled up, supplying an implementation of the Evaluator

12link// interface does the trick nicely

13linkpublic interface Evaluator {

14link  Boolean matches(Object calcItem);

15link}

16link

17link// refer to the "IsFlaggedBySalesUser" example above

18linkprivate static List<SObject> filter(List<SObject> calcItems, Evaluator eval) {

19link  List<SObject> applicableItems = new List<SObject>();

20link  for(SObject calcItem : calcItems) {

21link    if(eval != null && eval.matches(calcItem)) {

22link      applicableItems.add(calcItem);

23link    }

24link  }

25link  return applicableItems;

26link}

27link

28link// key the SObjects passed in to the String value

29link// matching the key on the object where the rollup

30link// will be performed

31linkprivate Map<String, List<SObject>> getCalcItemsByLookupField() {

32link  Map<String, List<SObject>> lookupFieldToCalcItems = new Map<String, List<SObject>>();

33link  for(SObject calcItem : this.calcItems) {

34link    String key = (String)calcItem.get(this.lookupFieldOnCalcItem);

35link    if(lookupFieldToCalcItems.containsKey(key) == false) {

36link      lookupFieldToCalcItems.put(key, new List<SObject>{ calcItem };

37link    } else {

38link      lookupFieldToCalcItems.get(key).add(calcItem);

39link    }

40link  }

41link  return lookupFieldToCalcItems;

42link}

43link

44link// your garden-variety dynamic SOQL

45linkprivate List<SObject> getLookupItems(Set<String> objIds) {

46link  String queryString =

47link    'SELECT Id, ' + this.lookupObjOpField.getDescribe().getName() +

48link    '\nFROM ' + this.lookupObj.getDescribe().getName() +

49link    '\nWHERE ' + this.lookupField.getDescribe().getName() + ' = :objIds';

50link  return Database.query(queryString);

51link}

52link

53link// the meat of the Rollup

54linkprivate void performRollup(Map<String, List<SObject>> calcItemsByLookupField, List<SObject> lookupItems) {

55link  List<SObject> toUpdate = new List<SObject>();

56link  for(SObject lookupRecord : lookupItems) {

57link    String key = (String)lookupRecord.get(this.lookupField);

58link    if(calcItemsByLookupField.containsKey(key) == false) {

59link      continue;

60link    }

61link

62link    List<SObject> calcItems = calcItemsByLookupField.get(key);

63link    Object priorVal = lookupRecord.get(this.lookupObjOpField);

64link    Object newVal = this.getRollupVal(calcItems, priorVal);

65link    lookupRecord.put(this.lookupObjOpField, newVal);

66link    toUpdate.add(lookupRecord);

67link  }

68link

69link  update toUpdate;

70link}

71link

72link// right now, this "works" - but we'll need to further generalize

73link// to support other kinds of rollup operations

74linkprivate Object getRollupVal(List<SObject> calcItems, Object priorVal) {

75link  Decimal returnVal = priorVal == null ? 0 : (Decimal)priorVal;

76link  for(SObject calcItem : calcItems) {

77link    switch on this.op {

78link      when SUM {

79link        returnVal += (Decimal)calcItem.get(this.opField);

80link      }

81link      when DELETE_SUM {

82link        returnVal -= (Decimal)calcItem.get(this.opField);

83link      }

84link      when UPDATE_SUM {

85link        Decimal oldVal = (Decimal)this.oldCalcItems.get(calcItem.Id).get(this.opField);

86link        Decimal newVal = (Decimal)calcItem.get(this.opField);

87link        returnVal += (newVal - oldVal); // could be negative, could be positive

88link      }

89link      when else {

90link        throw new IllegalArgumentException('Other rollup op: ' + this.op.name() + ' not yet implemented');

91link      }

92link    }

93link  }

94link  return returnVal;

95link}

The framework for rolling values up appears fairly quickly; this isn't really even that many lines of code. Indeed, the rest of the Rollup is largely defined by constructors at this point, and static methods exposing the rollup operations.

linkReduce, Reuse, Refactor: Rollup, Part Two

First things first -- that getRollupVal method needs to decouple itself from the type of rollup being performed. This is a great use-case for inner classes:

1linkprivate Object getRollupVal(RollupCalculator calc, List<SObject> calcItems, Object priorVal) {

2link  Rollup rollup = this.getRollupType(priorVal);

3link  for (SObject calcItem : calcItems) {

4link    rollup.performRollup(calc.op, priorVal, calcItem, calc.oldCalcItems, calc.opField.getDescribe().getName());

5link  }

6link  return rollup.getReturnValue();

7link}

8link

9linkprivate Rollup getRollupType(Object priorVal) {

10link  // fun fact - integers, doubles, longs, and decimals

11link  // will ALL return true here

12link  if (priorVal instanceof Decimal) {

13link    return new DecimalRollup(priorVal);

14link  } else {

15link    throw new IllegalArgumentException('Rollup operation not defined for: ' + JSON.serialize(priorVal));

16link  }

17link}

18link

19linkprivate abstract class Rollup {

20link  protected Object returnVal;

21link  public Rollup(Object returnVal) {

22link    this.returnVal = returnVal;

23link  }

24link  public Object getReturnValue() {

25link    return returnVal;

26link  }

27link  public abstract void performRollup(Op op, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, String operationField);

28link}

29link

30linkprivate class DecimalRollup extends Rollup {

31link  public DecimalRollup(Object priorVal) {

32link    super(priorVal == null ? 0 : priorVal);

33link  }

34link

35link  public override void performRollup(Op operation, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, String operationField) {

36link    Decimal returnVal = (Decimal) this.returnVal;

37link    switch on operation {

38link      when SUM {

39link        returnVal += (Decimal) calcItem.get(operationField);

40link      }

41link      when DELETE_SUM {

42link        returnVal -= (Decimal) calcItem.get(operationField);

43link      }

44link      when UPDATE_SUM {

45link        Decimal oldVal = (Decimal) oldCalcItems.get(calcItem.Id).get(operationField);

46link        Decimal newVal = (Decimal) calcItem.get(operationField);

47link        returnVal += (newVal - oldVal); // could be negative, could be positive

48link      }

49link      when else {

50link        throw new IllegalArgumentException('Other rollup op: ' + operation.name() + ' not yet implemented');

51link      }

52link    }

53link  }

54link}

If you're looking at getRollupType in the above example and thinking that it looks like the Factory pattern, that's a bingo! Now the type of rollup being performed has been decoupled from the logic necessary to perform the actual rollup. That makes it easy to add in:

A potential code smell (perhaps obscured by only having one rollup operation defined) is the switch statement in DecimalRollup. This piece of logic will have to be replicated in each Rollup inner class prior to us being able to proceed. However, with a slight shift in perspective comes an object-oriented opportunity to reduce the boilerplate necessary to introduce new rollup types into the mix:

1link// in Rollup.cls

2linkprivate enum Op {

3link  SUM,

4link  UPDATE_SUM,

5link  DELETE_SUM,

6link  COUNT, // our first new operation!

7link  UPDATE_COUNT,

8link  DELETE_COUNT

9link}

10link

11linkprivate Rollup getRollupType(Object priorVal, Op operationType) {

12link  // have to use the fully qualified Op name here (including the outer class)

13link  // since its type is shadowed in this method

14link  if(operationType.name().contains(RollupCalculator.Op.COUNT.name())) {

15link    return new CountRollup(priorVal);

16link  } else if (priorVal instanceof Decimal) {

17link    return new DecimalRollup(priorVal);

18link  } else {

19link    throw new IllegalArgumentException('Rollup operation not defined for: ' + JSON.serialize(priorVal));

20link  }

21link}

22link

23linkprivate abstract class Rollup {

24link  protected Object returnVal;

25link  public Rollup(Object returnVal) {

26link    this.returnVal = returnVal;

27link  }

28link  // we make this virtual to deal with downcasting

29link  public virtual Object getReturnValue() {

30link    return returnVal;

31link  }

32link  public abstract void performRollup(Op op, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, SObjectField operationField);

33link}

34link

35linkprivate virtual class DecimalRollup extends Rollup {

36link  public DecimalRollup(Object priorVal) {

37link    // much as it pains me to duplicate the null check, it must be done;

38link    // we can't reference instance methods till after the super() call

39link    super(priorVal == null ? 0 : priorVal);

40link  }

41link

42link  protected Decimal getDecimalOrDefault(Object potentiallyUnitializedDecimal) {

43link    return (Decimal) (potentiallyUnitializedDecimal == null ? 0 : potentiallyUnitializedDecimal);

44link  }

45link

46link  protected virtual Decimal getNumericValue(SObject calcItem, SObjectField operationField) {

47link    return this.getDecimalOrDefault(calcItem.get(operationField));

48link  }

49link

50link  protected virtual Decimal getNumericChangedValue(SObject calcItem, SObjectfield operationField, Map<Id, SObject> oldCalcItems) {

51link    Decimal newVal = this.getNumericValue(calcItem, operationField);

52link    Decimal oldVal = this.getNumericValue(oldCalcItems.get(calcItem.Id), operationField);

53link    // could be negative, could be positive ... could be 0!

54link    return newVal - oldVal;

55link  }

56link

57link  public override void performRollup(Op operation, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, SObjectField operationField) {

58link    Decimal returnVal = (Decimal) this.returnVal;

59link    switch on operation {

60link      when SUM, COUNT {

61link        returnVal += this.getNumericValue(calcItem, operationField);

62link      }

63link      when DELETE_SUM, DELETE_COUNT {

64link        returnVal -= this.getNumericValue(calcItem, operationField);

65link      }

66link      when UPDATE_SUM, UPDATE_COUNT {

67link        returnVal += this.getNumericChangedValue(calcItem, operationField, oldCalcItems);

68link      }

69link      when else {

70link        throw new IllegalArgumentException('Other rollup op: ' + operation.name() + ' not yet implemented');

71link      }

72link    }

73link  }

74link}

75link

76linkprivate class CountRollup extends DecimalRollup {

77link  public CountRollup(Object priorVal) {

78link    super(priorVal);

79link  }

80link

81link  public override Object getReturnValue() {

82link    return (Integer) this.returnVal;

83link  }

84link

85link  protected override Decimal getNumericValue(SObject calcItem, SObjectField operationField) {

86link    Decimal potentialReturnValue = super.getNumericValue(calcItem, operationField);

87link    return this.getCountValue(potentialReturnValue);

88link  }

89link

90link  protected override Decimal getNumericChangedValue(SObject calcItem, SObjectField operationField,

91link    Map<Id, SObject> oldCalcItems) {

92link    Decimal potentialReturnValue = super.getNumericChangedValue(calcItem, operationField, oldCalcItems);

93link    return this.getCountValue(potentialReturnValue);

94link  }

95link

96link  private Decimal getCountValue(Decimal potentialReturnValue) {

97link    return potentialReturnValue > 0 ? 1 : potentialReturnValue;

98link  }

99link}

Et voila -- in just a few short lines and an additional check in the factory method, we've added a completely different rollup type into the mix. The new CountRollup class has an interesting condition to it (consistent with the rules for how the COUNT function works in SOQL) -- it still requires there to be a non-null value on the field within the calculation item that it's comparing to. It may be the case that people simply want to count all child/related objects and roll up their presence to a parent/related object. I'll be curious to hear from you as to whether or not that functionality is desired, prior to implementing a so-called BlindCount version.

As I added more functionality, it occurred to me that either by wanting to perform many rollups from within a single trigger, or by having rollup operations involving large amounts of records, it might be possible to creep towards the synchronous DML limit of 10,000 DML rows in a single transaction. Making Rollup perform the bulk of the work async also led to implementing the Data Processor pattern -- by burning a few SOQL queries on the synchronous side, Rollup can automatically (or through the use of RollupLimit__mdt custom metadata) scale from running as a Queueable to a Batchable as the size of rollup operations grow. This is the power of elastic scaling!

linkRollup - A Note On Progress

I wrote the above paragraph on December 21st, and jotted down a note to myself:
TODO:
max/min ??.

Today is December 29th. It's not that I took a break from writing this article; I didn't. What happened? Over the hours (and then days) that followed, I began to add additional operations for Rollup. My assumption -- with the SUM operations for numbers basically "complete" -- surrounding the ease of implementing the rest of the rollup functions slowly began to wither on the vine. I laughed on Christmas when a stranger submitted a PR on one of my open-source repos -- one thing I was no stranger to was putting in long hours on passion projects, and somebody else was clearly taking advantage of the holidays.

What began as a casual foray into adding COUNT quickly began to escalate outwards as the code smell that I mentioned earlier -- the switch statement in DecimalRollup became bigger and bigger.

"This isn't really even that many lines of code" - a younger, more naive version of me (earlier in this article), before the switch statement in DecimalRollup ended up as 50% of the size of the entire Rollup class when I started

The addition of an Invocable entry point for Flows and Process Builders contributed to the rising tide of lines of code -- and at the end of that journey, without many of the rollup functions even implemented yet, I realized it was past time ... the tests needed to be ported over, and new ones created ASAP to ensure what I had so far was going to work.

linkDesign Decisions For Testing The Rollup Framework

If you've read The Joys Of Apex before, you know that keeping your Salesforce tests running quickly is something I'm passionate about. This was doubly important in the current context: the tests I'd originally written were tightly coupled to two things that weren't going to make the cut:

One of the reasons that DLRS is so big as a codebase is because it imports a ton of code from FFLib to perform the metadata deployments necessary to create DLRS triggers/rollups as needed. It occurred to me that I was trying to encourage something more along the lines of adding an analytics tag to a website than the sort of service DLRS offerred: I've worked on several analytics implementations, and vetted many vendors. I've never heard of somebody offering a service that didn't require the installation of a JavaScript / backend SDK in order to work (even Cloudflare and other reverse proxy services require up-front configuration). In other words, as I considered how I wanted my tests to work, I had to also solidify the concepts (like Rollup being more akin to installing Google Analytics) necessary for Rollup to be used.

Since my prior tests were a no-go, I found myself recalling the immortal words of Kent Beck in "Test Driven Development By Example":

You will often be implementing TDD in code that doesn't have adequate tests. When you don't have enough tests, you are bound to come across refactorings that aren't supported by tests ... what do you do? Write the tests you wish you had.

In order to make the tests run fast, I needed to limit the amount of DML performed. Crucial to that effort would be the following class and seam:

1link// in Rollup.cls

2link@testVisible

3linkprivate virtual class DMLHelper {

4link  public virtual void doUpdate(List<SObject> recordsToUpdate) {

5link    update recordsToUpdate;

6link  }

7link}

8link

9link/**

10link  * receiving an interface/subclass from a property get/set (from the book "The Art Of Unit Testing") is an old technique;

11link  * useful in limited contexts to get around the classic approach to dependency injection

12link  * (such as in this case, when constructor-based DI isn't possible).

13link  * It's more palatable in Apex than in many other languages, as a matter of fact -

14link  * this is because the @testVisible annotation enforces for us the override only being possible while testing

15link  */

16link@testVisible

17linkprivate static DMLHelper DML {

18link  get {

19link    if (DML == null) {

20link      DML = new DMLHelper();

21link    }

22link    return DML;

23link  }

24link  set;

25link}

26link

27link// and then in the tests:

28link

29linkprivate class DMLMock extends Rollup.DMLHelper {

30link  public List<SObject> Records = new List<SObject>();

31link  public override void doUpdate(List<SObject> recordsToUpdate) {

32link    this.Records = recordsToUpdate;

33link  }

34link}

Wait a minute. Doesn't this look familiar? It should. But because I was intent on keeping everything within one class, I couldn't import dependencies -- I either needed to recreate them inside of the Rollup class, or do without them. The Factory pattern for dependency injection and the Repository pattern for strongly-typed and easily mocked queries were both discarded as a result. That left me with the DML Mock pattern -- and sure enough, all of the tests are lightning-fast as a result. Here's a simple one:

1link// in RollupTests.cls

2link@isTest

3linkstatic void shouldSumFromTriggerAfterInsert() {

4link  DMLMock mock = getMock(new List<Opportunity>{ new Opportunity(Amount = 25), new Opportunity(Amount = 25) });

5link  Rollup.triggerContext = TriggerOperation.AFTER_INSERT;

6link

7link  Rollup rollup = Rollup.sumFromTrigger(

8link    Opportunity.Amount,

9link    Opportunity.AccountId,

10link    Account.Id,

11link    Account.AnnualRevenue,

12link    Account.SObjectType

13link  );

14link

15link  System.assertEquals(true, mock.Records.isEmpty());

16link

17link  Test.startTest();

18link  rollup.runCalc();

19link  Test.stopTest();

20link

21link  System.assertEquals(1, mock.Records.size(), 'Records should have been populated SUM AFTER_INSERT');

22link  Account updatedAcc = (Account) mock.Records[0];

23link  System.assertEquals(50, updatedAcc.AnnualRevenue, 'SUM AFTER_INSERT should add the original opportunity amount');

24link}

One hidden side-effect is the tying of the Opportunities to the Account in question - that happens in the getMock method:

1link// in RollupTests.cls

2linkprivate static DMLMock getMock(List<SObject> records) {

3link  Account acc = [SELECT Id FROM Account];

4link  for (SObject record : records) {

5link    record.put('AccountId', acc.Id);

6link  }

7link

8link  return loadMock(records);

9link}

10link

11link// ...

12link

13linkprivate static DMLMock loadMock(List<SObject> records) {

14link  Rollup.records = records;

15link  Rollup.shouldRun = true;

16link  DMLMock mock = new DMLMock();

17link  Rollup.DML = mock;

18link

19link  return mock;

20link}

There are, in fact, only three other helper methods (non test methods) in the entire test class. Once I had tests that covered the basics of what I'd written for the SUM and COUNT implementations, it was time to approach this thing TDD-style:

This simple rhythm helped me to immediately spot a flaw in the code for the Invocable method as I worked to create tests for this new functionality -- it was being fed into a method that used the current Trigger context (the TriggerOperation enum) to figure out what kind of rollup operation was being performed. That's also where the Rollup.records variable came from in the loadMock code, above; a way to stub in the trigger records without actually having to require a trigger being run.

When you look at the first ~20 lines of Rollup, you can see the use of these @testVisible private static variables as the "poor man's dependency injection:"

1link  /**

2link   * Test override / bookkeeping section. Normally I would do this through dependency injection,

3link   * but this keeps things much simpler

4link   */

5link  @testVisible

6link  private static Boolean shouldRun;

7link  @testVisible

8link  private static Boolean shouldRunAsBatch = false;

9link  @testVisible

10link  private static TriggerOperation triggerContext = Trigger.operationType;

11link  @testVisible

12link  private static Map<Id, SObject> oldRecordsMap;

13link  @testVisible

14link  private static List<Rollup__mdt> rollupMetadata;

15link  @testVisible

16link  private static List<SObject> queryRecords;

17link  @testVisible

18link  private static RollupLimit__mdt defaultRollupLimit;

19link  @testVisible

20link  private static RollupLimit__mdt specificRollupLimit;

There are some juicy hints, above, of what was ultimately to come.

linkAdding Custom Metadata-driven Rollups

Adding the CMDT-record driven rollups was simple now that I had a burgeoning test suite and two different possible points of entry (Invocable / Trigger-based) into Rollup. Indeed, because the test suite was expansive and I believe that CMDT rollups form the core of peoples' rollup needs, refactoring the code to support custom metadata as a first-class citizen became (truly) a joy. You can tell that the code is really built around it because there are only 3 lines of code in the public-facing method:

1link// in Rollup.cls - don't mind that null argument below, it's for the custom Evaluator interface

2linkpublic static void runFromTrigger() {

3link  SObjectType sObjectType = getTriggerRecords().getSObjectType();

4link  List<Rollup__mdt> rollupMetadata = getTriggerRollupMetadata(sObjectType);

5link  runFromTrigger(rollupMetadata, null).runCalc();

6link}

This is the "Google Analytics" approach: in order to use Rollup, all you need to do is add one line of code to your triggers:

1linkRollup.runFromTrigger();

Unless you need a ton of customization, it's really as simple as that. Please note: this requires your trigger to use the following contexts: after insert, after update, and before delete. Without those in place, Rollup will not function as designed for trigger-based rollups!

The rest of the info -- about which rollups need to be processed for the trigger in question -- can all live in the Rollup__mdt Custom Metadata:

There are some peculiarities within Apex when working with Entity Definition and Field Definition-based Custom Metadata fields, which I will detail just below in the key takeaways section!

linkInvoking Rollup.cls From A Process Builder / Flow

Invoking the `Rollup` process from a Flow, in particular, is a joy; with a Record Triggered Flow, you can do the up-front processing to take in only the records you need, and then dispatch the rollup operation to the `Rollup` invocable:

Example flow

This is also the preferred method for scheduling; while I do expose the option to schedule a rollup from Apex, I find the ease of use in creating Scheduled Flows in conjunction with the deep power of properly configured Invocables to be much more scalable than the "Scheduled Jobs" of old. This also gives you the chance to do some truly crazy rollups -- be it from a Scheduled Flow, an Autolaunched Flow, or a Platform Event-Triggered Flow. As long as you can manipulate data to correspond to the shape of an existing SObject's fields, they don't even have to exist; you could have an Autolaunched flow rolling up records when invoked from a REST API so long as the data you're consuming contains a String/Id matching something on the "parent" rollup object.

linkKey Takeaways In Replacing DLRS

There were quite a few learning moments as I worked through rollup edge cases; I've chosen to spend the rest of this article articulating ones that I think might be helpful or interesting to you in your own Salesforce journey:

linkEntity Definition & Field Definition Custom Metadata Relationships Can Be Tricky

There's not a whole lot of documentation out there about Entity / Field Definition-basd CMDT. They are as good as they sound -- giving users of your CMDT object-level and field-level safety when letting them select fields, but something interesting that I found while working with Field Definition fields in Apex is that they are stored as "ObjectName.FieldName" in the database. This roughly corresponds to a string-level representation of what an SObjectField type is written as:

1link// if we query for the Rollup__mdt shown earlier

2linkRollup__mdt rollupMetadata = [SELECT RollupFieldOnLookupObject__c FROM Rollup__mdt LIMIT 1];

3linkSystem.debug(rollupMetadata.RollupFieldOnLookupObject__c); // outputs "Opportunity.Amount", for example

4linkSObjectField opportunityAmount = Opportunity.Amount;

5linkSystem.debug(opportunityAmount); // ouputs ... "Amount" ... so you know that somebody overrode the "toString()" method for this class!

When working with dynamic fields in SOQL, developers frequently use the DescribeSObjectResult and DescribeFieldResult classes that give you access to metadata about objects/fields ... but in this case, I found I had to create a helper method specifically for working with the String-based version of the Field Definition values coming in from the Rollup__mdt records:

1link// takes a string from CMDT like "Opportunity.Amount" and returns just the field name: "Amount"

2link// this allows us to match the String-based version of the field with its corresponding SObjectField

3link// by calling describeForSObject.fields.getMap().get(theFieldNameReturnedFromgetParedFieldName)

4linkprivate static String getParedFieldName(String fullFieldName, DescribeSObjectResult describeForSObject) {

5link  return String.isBlank(fullFieldName) ? '' : fullFieldName.replace(describeForSObject.getName() + '.', '');

6link}

linkUsing Enums Is Great, But Instantiating Them From Strings Isn't Obvious

You might remember from the Apex Enum Class Gotchas article that sending the name() value for an enum is the only way to properly deserialize the enum when you're ingesting data either in Apex or in another service. That's all well and good -- but, as it turns out, you can't deserialize directly to the String-based enum:

1link// in Rollup.cls - making this a public enum doesn't change the result

2linkprivate enum Op {

3link  SUM,

4link  UPDATE_SUM,

5link  DELETE_SUM

6link  // etc

7link}

8link

9linkRollup__mdt rollupMetadata = methodWhereWeGetTheMetadata();

10link// this is a crazy thing and I wouldn't have wanted to do it anyway, but science ...

11link// if you ACTUALLY need to do these things from within Apex, I highly recommend the use of the JSONGenerator class

12linkString operationFromMetadata = '{ "op" : "' + rollupMetadata.RollupType__c + '"}';

13linkOp operation = (Op)JSON.deserialize(operationFromMetadata, Op.class);

14linkSystem.debug(operation);

15link// outputs null

16link

17link// you COULD make a wrapper class

18linkprivate class OpWrapper {

19link  public Op op { get; set; }

20link}

21linkOpWrapper opWrapper = (OpWrapper)JSON.deserialize(operationFromMetadata, OpWrapper.class);

22linkSystem.debug(opWrapper);

23link// outputs: "OpWrapper:[Op=SUM]", for example

Keying the string from the CMDT to be valid JSON and creating a wrapper class left me feeling a little ill, so instead I went with a lazily-loaded Map<String, Op> using the included values() method present on all enums:

1linkprivate static Map<String, Op> opNameToOp {

2link  get {

3link    if (opNameToOp == null) {

4link      opNameToOp = new Map<String, Op>();

5link      for (Op operation : Op.values()) {

6link        opNameToOp.put(operation.name(), operation);

7link      }

8link    }

9link    return opNameToOp;

10link  }

11link  set;

12link}

It's not perfect, but it's better than crafting JSON to get around not being able to cast from a String to an enum, and not being able to reference the enum by its name() in any other way.

linkNot all fields of the same type in Salesforce support MIN or MAX operations

This is probably true of other field types (honestly, at this point, nothing would really surprise me), but it came up frequently for me while testing with Date / Datetime fields, as well as multi-select picklists (the Devil's picklists, some would say).

It did come as a real surprise to me as I was writing tests; I had used only the Account and Opportunity objects so far in my tests, in an effort to keep them as generic as possible. I wanted to write a test using a Date field that wasn't required on insert to test my DefaultFieldInitializer (more on that in a second). While you can MIN or MAX the Opportunity's CloseDate field, the "parent" object I had been using as the target of my rollups, Account, didn't have a Date field on it. In retrospect, I could have used the Contract object (and later I would), but I'm glad I didn't, as I might not have encountered this charming error message otherwise: System.QueryException: There's a problem with your query: field ActivityDate does not support aggregate operator MAX. Uhhh, OK. Thanks for that, Salesforce.

Wherever possible, I've tried to make the code resilient to the idiosyncracies of the platform; for operations like this, if that means doing the damn MIN / MAX myself, so be it:

1link// MIN/MAX is allowed, but not for all fields, and not consistently. Go figure!

2linkprotected virtual override Object calculateNewAggregateValue(Set<Id> excludedItems, Op operation, SObjectField operationField, SObjectType sObjectType) {

3link  Object aggregate;

4link  try {

5link    aggregate = super.calculateNewAggregateValue(excludedItems, operation, operationField, sObjectType);

6link  } catch (Exception ex) {

7link    // technically a System.QueryException, but I figure we might as well catch em all and try like hell to aggregate anyway

8link    Decimal minOrMax;

9link    List<SObject> allOtherItems = Database.query('SELECT ' + operationField + ' FROM ' + sObjectType + ' WHERE Id != :excludedItems');

10link    for (SObject otherItem : allOtherItems) {

11link      Decimal otherItemDate = this.getDecimalOrDefault(otherItem.get(operationField));

12link      if (otherItemDate != null && operation.name().contains(Op.MAX.name()) && (minOrMax == null || otherItemDate > minOrMax)) {

13link        minOrMax = otherItemDate;

14link      } else if (otherItemDate != null && operation.name().contains(Op.MIN.name()) && (minOrMax == null || otherItemDate < minOrMax)) {

15link        minOrMax = otherItemDate;

16link      }

17link    }

18link    if (minOrMax == null) {

19link      aggregate = operation.name().contains(Op.MIN.name()) ? FieldInitializer.maximumLongValue : FieldInitializer.minimumLongValue;

20link    } else {

21link      aggregate = minOrMax;

22link    }

23link  }

24link

25link  return aggregate;

26link}

27link// not pictured - the "hot as hell" section where min/max gets tabulated for multi-select picklists. Yowza! Feel the burn!

linkNull as a value doesn't retain its type information

I thought I'd be able to simply query a given field on the rollup object in question, test for its type using instanceof, and then move on to rolling up. Unfortunately, that approach failed on my very first test where the rollup object didn't have the rollup field initialized. I needed to find something -- anything -- that would give me a clue about what the type was for a given field at runtime, and found it in the DisplayType enum. Indeed, I found out after creating the default value initializer that this was the same method employed by a number of dynamic Apex test libraries:

1link// in Rollup.cls

2linkprivate virtual class DefaultSObjectFieldInitializer {

3link  public final Datetime defaultDateTime = Datetime.newInstanceGmt(1970, 1, 1);

4link  public final Long maximumLongValue = (Math.pow(2, 63) - 1).longValue();

5link  public final Long minimumLongValue = this.maximumLongValue * -1;

6link

7link  public virtual Object getDefaultValue(SObjectField field) {

8link      DescribeFieldResult fieldDescribe = field.getDescribe();

9link      if (fieldDescribe.isDefaultedOnCreate()) {

10link        return fieldDescribe.getDefaultValue();

11link      }

12link      // not surprisingly, "getDefaultValue" on the DescribeFieldResult returns null for fields without default values

13link      // this is a shame - all types *should* have default values. Instead, we have the privilege of getting to initialize them

14link      Object initializedDefault;

15link      switch on fieldDescribe.getType() {

16link      when CURRENCY, DOUBLE, INTEGER, LONG, PERCENT {

17link        initializedDefault = 0;

18link      }

19link      when DATETIME {

20link        initializedDefault = this.defaultDateTime;

21link      }

22link      when DATE {

23link        initializedDefault = this.defaultDateTime.dateGmt();

24link      }

25link      when TIME {

26link        initializedDefault = this.defaultDateTime.timeGmt();

27link      }

28link      when STRING, ID, TEXTAREA, URL, PHONE, EMAIL{

29link        initializedDefault = '';

30link      }

31link      when PICKLIST, MULTIPICKLIST {

32link        // more on this part in a second

33link        initializedDefault = new PicklistController(field.getDescibe()).getDefaultValue(field);

34link      }

35link      when else {

36link        throw new IllegalArgumentException('Field: ' + field + ' of type: ' + fieldType.name() + ' specified invalid for rollup operation');

37link      }

38link    }

39link    return initializedDefault;

40link  }

41link}

linkFinding default SObject fields for different DisplayTypes is fun

In the end, this Anonymous Apex script proved invaluable for hunting down standardly available fields of special types like TIME:

1linkpublic static void printDisplayTypeInfo(DisplayType desiredType) {

2link  Map<String, SObjectType> namesToTypes = Schema.getGlobalDescribe();

3link  for(SObjectType sType : namesToTypes.values()) {

4link    Map<String, SObjectField> fields = sType.getDescribe().fields.getMap();

5link    for(String fieldName : fields.keyset()){

6link      SObjectField field = fields.get(fieldName);

7link      DescribeFieldResult describeResult = field.getDescribe();

8link      if(describeResult.getType() == desiredType && describeResult.isUpdateable()) {

9link          System.debug('SObjectType: ' + sType);

10link          System.debug(describeResult.getName());

11link          // you could put a return statement here to only print the first result found

12link      }

13link    }

14link  }

15link}

I had never worked with the ContactPointAddress or ContactPointEmail objects before; they're relatively new additions to the system, and it was fun to learn a bit more about them while using them to wire up different zany relationships.

linkObject-Oriented Programming Is Extremely Powerful (All Dates Are Numbers)

I've talked quite a bit about the infamous switch statement now in the DecimalRollupCalculator. After implementing all of the rollup operations for numbers, I teetered at the precipice -- how to take the logic in DecimalRollupCalculator and generalize it so that other subclasses could make use of it. As I considered the horrors of a switch statement that needed to differentiate not only between different rollup operations, but also their context (was it an insert? an update? a delete??), my stomach began to twist. I had just written my first failing test for implementing MAX for Datetimes. Would my response really be to copy pasta?

Then it hit me. Datetimes are all stored uniformly within Salesforce in UTC time. UTC can be represented by numbers. Dates are just Datetimes with a zero'd out Time section. The game was on. In the end, this section truly took very little additional code to get right:

1link// omitting the Datetime parent class, which itself descends from DecimalRollupCalculator

2link// the reason should be clear if you take a peek at the source code. While it's simple, an

3link// excerpt from DatetimeRollupCalculator is shown below in the "SOQL Drops Milliseconds From Datetimes ..." section

4linkprivate class DateRollupCalculator extends DatetimeRollupCalculator {

5link    // for Date, it's not necessary to override the "getDecimalOrDefault" method in DatetimeRollupCalculator

6link    // because the conversion only happens in "getReturnValue"

7link    public DateRollupCalculator(Object priorVal, SObjectField operationField) {

8link      super(Datetime.newInstanceGmt((Date) priorVal, Time.newInstance(0, 0, 0, 0)), operationField);

9link    }

10link

11link    public override Object getReturnValue() {

12link      return ((Datetime) super.getReturnValue()).dateGmt();

13link    }

14link  }

15link

16link  private class TimeRollupCalculator extends DatetimeRollupCalculator {

17link    public TimeRollupCalculator(Object priorVal, SObjectField operationField) {

18link      super(Datetime.newInstanceGmt(FieldInitializer.defaultDateTime.dateGmt(), (Time) priorVal), operationField);

19link    }

20link

21link    public override Object getReturnValue() {

22link      return ((Datetime) super.getReturnValue()).timeGmt();

23link    }

24link

25link    protected override Decimal getDecimalOrDefault(Object potentiallyUnitializedDecimal) {

26link      Datetime defaultDatetime;

27link      if (potentiallyUnitializedDecimal instanceof Time) {

28link        defaultDatetime = Datetime.newInstanceGmt(FieldInitializer.defaultDateTime.dateGmt(), (Time) potentiallyUnitializedDecimal);

29link      } else if (potentiallyUnitializedDecimal instanceof Decimal) {

30link        defaultDatetime = Datetime.newInstance(((Decimal) potentiallyUnitializedDecimal).longValue());

31link      } else {

32link        defaultDatetime = FieldInitializer.defaultDateTime;

33link      }

34link      return defaultDatetime.getTime();

35link    }

36link  }

That commit -- and the powerful realization that I'd been able to add a possible sixty-three different permutations for rollup operations in what amounted to a mere ~32 lines of code ... that's the power of Object-Oriented Programming!

Edit -- what I failed to mention, upon originally publishing this article, was that there was a duplicated switch statement (for String-based rollups); a detail that had bothered me in the days leading up to release, but which I didn't have the time (or energy, after several marathon days spent "finishing" Rollup) to address prior to launching. Several days after releasing, I went back and made use of the Chain Of Responsibility to break up this duplication. As is common when virtual methods are added at the parent level, this actually ended up increasing the lines of code in the final Rollup file. At the same time, indentation (one of the great sins present in the use of switch statements) was much reduced. What was left? One switch statement to rule them all, with virtual methods to bind them. While lines of code can be a metric for complexity, it also often ends up being deceptive -- the code reads more like a story, now, where rollup operations are explicitly opted into, instead of having to scan through a switch statement to determine which operation leads to which result.

linkintValue() on a Long can be a wild ride

While working on MAX/MIN based code, I realized quickly that there were only two reliable sentinel values when it came to numbers -- the maximum possible number that would fit into a 64-bit number (a Long), and the minimum possible number. What bit me really, really hard while working on implementing min/max for picklists? To anwer that, first an aside -- min/maxing on a picklist is supposed to return the deepest possible entry in the picklist (for MAX), or the closest to the top of the picklist (for MIN). Picklists have many ... interesting ... subleties in Salesforce, and the implicit concept of "rank" (just look at the Path components for Lead Status or Opportunity Stage Name) is just one of their many quirks.

If a value doesn't exist on the lookup object, that value should always lose; on a MIN it should be greater than the "rank" of any other field (so that any comparison to it leads to a truthy "less than" evaluation); on a MAX it should be less than the rank of any other field (likewise; it should lead to a truthy "greater than" evaluation). Making an object to perform these evaluations for picklists was a great exercise in Object Oriented Programming -- frequently throughout this project, that proved to be the case. Making the PicklistController inner class descend from the DefaultFieldInitializer made sense in the context of the object hierarchy. Though the PicklistController had many more responsibilities, it could also handle setting the default value for a picklist with ease.

Where things finally took a turn for the worse was when my first picklist test for ensuring MIN/MAX was working correctly failed. The logic looked perfect -- what could be the issue? It took me a painful 30 minutes to see the issue, so deeply ingrained were my assumptions about the way that Long values would translate to Integers. Only at the last second did I truly comprehend that I had betrayed myself:

1linkprivate class PicklistController extends DefaultSObjectFieldInitializer {

2link// etc ...

3link

4link  private Integer getSentinelValue(Boolean isMin) {

5link    return (isMin ? this.maximumLongValue : this.minimumLongValue).intValue();

6link  }

7link}

Passing Boolean values as method arguments is always to be strongly discouraged, but this method call was already on the tail end of a ternary and I felt, when creating it, that I had no other option. I will say, though, that I once had a coworker who was deeply passionate about using nested ternaries. I lost track of him over the years, but ... perhaps the issue would have been clearer to me if the culprit had been in a truly doubled ternary; after all, who in their right minds could avoid investigating such a travesty?

In any case. Do you know what the integer values are for the minimum and maximum Long values (-2^63 and 2^63, respectively)?

1linkLong maximumLongValue = (Math.pow(2, 63) - 1).longValue();

2linkLong minimumLongValue = maximumLongValue * -1;

3link

4linkSystem.debug(maximumLongValue); // prints: 9223372036854775807

5linkSystem.debug(minimumLongValue); // prints: -9223372036854775807

6link

7link// Now as ints!

8linkSystem.debug(maximumLongValue.intValue()); // prints -1

9linkSystem.debug(minimumLongValue.intValue()); // prints 1

Ouch. Well, that explained my failing test! My understanding (after a brief foray into the details) is that casting to Integer or calling intValue() is safe within the bounds of the 32-bit allowed integer sizes, but after that all bets are off. Creating the min/max integer bounds did the trick wonderfully. Running into edge cases like this can hurt, but it also expands the mind -- you only get bit by something like this once before knowing to look out for it next time around. Plus, I ended up being able to get rid of the Boolean passing altogether -- two birds with one stone!

linkSOQL drops the milliseconds from Datetime fields when they are retrieved from the database

This was a fun one -- and one that I was already aware of from lurking on the SFXD Discord. This is a real pain -- especially in testing --, but in the end the following bit did the trick nicely:

1link// one of the worst things about SOQL is that Datetimes retrieved have the millisecond values truncated

2linkDatetime datetimeWithMs = potentiallyUnitializedDecimal instanceof Decimal

3link  ? Datetime.newInstance(((Decimal) potentiallyUnitializedDecimal).longValue())

4link  : ((Datetime) potentiallyUnitializedDecimal);

5link// reading through the source code provides a more cogent rationale

6link// for the above eyesore over anything I can muster here.

7linkreturn Datetime.newInstanceGmt(

8link    datetimeWithMs.yearGmt(),

9link    datetimeWithMs.monthGmt(),

10link    datetimeWithMs.dayGmt(),

11link    datetimeWithMs.hourGmt(),

12link    datetimeWithMs.minuteGmt(),

13link    datetimeWithMs.secondGmt()

14link  )

linkCustom Rollup Wrap-up

Well, it's out there now. This article -- and the corresponding code -- has consumed an enormous quantity of time since work began on it in earnest in early December. I would highly recommend developers check out the source code (and the travelogue style commit history). Over the coming months I plan to add more functionality to Rollup -- for now, I'm hopeful that you'll consider trying it out. It's efficient, scales elastically, allows for rollups on fields (like Task.ActivityDate) that don't always allow for rollups in SOQL, and is well-tested.

I'm aware that Rollup doesn't hit 100% feature-parity versus DLRS ... and though I have plans to meet that challenge, as well, I believe that we're well past the fabled "80% of the functionality" stage. If your org (like many out there) is struggling under the weight of DLRS' auto-spawned triggers, I'm confident that Rollup will be a valuable tool for both the declaratively-minded as well as the developers out there.

linkPostscript

My original intent was to finish this article by December 27th, the one year anniversary of the Joys Of Apex. Despite some insanely long days spent writing, that didn't quite happen. Despite that, I just wanted to say that the readers of this series have helped ease the burden in a year that was extremely challenging for many people. Stuck inside for large portions of time, I took to writing -- and it shows. More than ten of the articles I wrote over the last year came out during the first 2 months of the pandemic -- some only days apart from one another. I do not advertise, and have not attempted to monetise in any way the incredible surge of traffic my personal website has experienced as a result. My intent is to provide readers with free content and materials to refer back to. Indeed, the only work I've done on the site over the past year was immediately preceeding this article, as I spruced up the Joys Of Apex blog page to better show off the posts.

All of that is to say -- thank you for an incredible year. Here's to hoping that 2021 will prove a better year for the world, and for you.

The original version of Replacing DLRS With Custom Rollup can be read on my blog.

Use Cases For Custom RollupsIntroducing RollupGetting The SObjectType From An SObjectFieldImplementing Rollups In ApexReduce, Reuse, Refactor: Rollup, Part TwoRollup - A Note On ProgressDesign Decisions For Testing The Rollup FrameworkAdding Custom Metadata-driven RollupsInvoking Rollup.cls From A Process Builder / FlowKey Takeaways In Replacing DLRSEntity Definition & Field Definition Custom Metadata Relationships Can Be TrickyUsing Enums Is Great, But Instantiating Them From Strings Isn't ObviousNot all fields of the same type in Salesforce support MIN or MAX operationsNull as a value doesn't retain its type informationFinding default SObject fields for different DisplayTypes is funObject-Oriented Programming Is Extremely Powerful (All Dates Are Numbers)intValue() on a Long can be a wild rideSOQL drops the milliseconds from Datetime fields when they are retrieved from the databaseCustom Rollup Wrap-upPostscript

Home Advanced Logging Using Nebula Logger Apex Logging Service Apex Object-Oriented Basics Batchable And Queueable Apex Building A Better Singleton Continuous Integration With SFDX Dependency Injection & Factory Pattern Enum Apex Class Gotchas Extendable Apis Formula Date Issues Future Methods, Callouts & Callbacks Idiomatic Salesforce Apex Introduction & Testing Philosophy Lazy Iterators Lightweight Trigger Handler LWC Composable Modal LWC Composable Pagination LWC Custom Lead Path Mocking DML React Versus Lightning Web Components Refactoring Tips & Tricks Replacing DLRS With Custom Rollup Repository Pattern setTimeout & Implementing Delays Sorting And Performance In Apex Test Driven Development Example Testing Custom Permissions The Tao Of Apex Writing Performant Apex Tests



Read more tech articles