Replacing DLRS With Custom Rollup
./img/joys-of-apex-thumbnail.png
A few months ago I was tasked with replacing Declarative Lookup Rollup Summaries (DLRS) in an org suffering from frequent deadlocks. Rollup summary fields in Salesforce are plagued by severe limitations -- only being available on master-detail relationships being just the start of the list. When faced with implementing custom rollups, most people go with DLRS because it's free. Performance in the org inevitably suffers.
Read on to learn about how I built Rollup
, complete with elastic scaling (go fast when you need to, slow when there are more rollups to process), to assist in orgs looking for DLRS-like flexibility with a much smaller performance overhead.
Discover how the "Google Analytics" approach to implementing rollups in your Salesforce org gives you the flexibility and power that you need - install with:
OR opt-into truly powerful code-based hooks that allow you to perform custom-based filtering using a simple Evaluator
interface.
Keep reading, or check out the repository Readme for more!
Let's look at some of the problem areas experienced when implementing rollup fields:
TotalOfGroupA__c
and TotalOfGroupB__c
)The list could go on and on. None of these use cases are supported with out-of-the-box Salesforce. You can see in many areas of the platform that providing developers with ways to augment the existing behavior for out-of-the-box features leads to awesome customizations. With rollups, we don't have the ability through the UI to specify something like an overriding Apex class; we have no interface to implement. No -- if we want something better, we have to build it ourselves.
With the stage set and the problem well-defined, let's take a look at the beginnings of the Rollup
project by examining how to invoke it from within a Trigger Handler class:
1link// within the "MyCustomObject__c" trigger handler
2link
3link// I'm aware it's not the sexiest interface. Keeping track of which field is which
4link// is now engraved in my head, but I'll annotate for now
5link// I expect most people will be using CMDT to manage these fields, anyway
6link
7linkRollupCalculator.sumFromTrigger( // the rollup operation
8link MyCustomObject__c.Amount__c, // the field that will inform the rollup
9link MyCustomObject__c.OwnerId, // the field on MyCustomObject related to the next argument, the matching field on the related object
10link User.Id, // the matching field on the object where the rollup will be performed
11link User.TotalCustomAmount__c, // the field where the rollup will be written to
12link new IsFlaggedBySalesUser(), // optional - a way to filter the "newCustomObjects" for ones that match
13link User.SObjectType // the related object where the rollup will be informed
14link);
15link
16linkprivate class IsFlaggedBySalesUser implements Rollup.Evaluator {
17link public Boolean matches(Object calcItem) {
18link MyCustomObject__c customObj = (MyCustomObject__c)calcItem;
19link return customObj.IsFlaggedBySalesUser__c;
20link }
21link}
In order to create something within Apex that can capture everything necessary for performing rollups in a generic way, many arguments are required. You could get rid of that last argument -- the SObjectType
-- but, sadly, there is no method on the existing DescribeFieldResult class that points to the object it was initialized from.
There is a workaround that's been floating around the Salesforce Stack Exchange for several years:
1linkpublic static SObjectType getSObjectType(Schema.SObjectField field) {
2link // This is a solution that was proposed on the Salesforce stack exchange
3link // and is the only work-around to a native lookup
4link // that I have been able to find.
5link Integer fieldHash = ((Object)field).hashCode();
6link
7link // Build a map of hashcodes for each fieldDescribe token
8link Map<String, Schema.SObjectType> globalDescribe = Schema.getGlobalDescribe();
9link Map<Integer,Schema.SObjectType> fieldHashCodeToSObjectTypeMap = new Map<Integer,Schema.SObjectType>();
10link for (String sobjname: globalDescribe.keySet()) {
11link for (Schema.SObjectField sobjField : globalDescribe.get(sObjName).getDescribe().fields.getMap().values())
12link fieldHashCodeToSObjectTypeMap.put(((Object) sObjField).hashCode(), globalDescribe.get(sobjName));
13link }
14link
15link // hard to believe it, but this actually works! it's a testament to the Describe objects, really:
16link // it means that any SObjectField token is a singleton! still, I would NEVER use this in production-level code
17link return fieldHashCodeToSObjectTypeMap.get(fieldHash);
18link}
Yikes. While that works, it falls down when you want to further generalize; it only works with strongly-typed SObjectField
s; while the use of SObjectField
types is a great experience for the developer, they too have shortcomings -- like not supporting parent or child relationships. While it's perfectly valid, using the MyCustomObject__c
SObject from above, to reference MyCustomOject__c.Owner.Name
, for example, you cannot construct an SObjectField
to represent that relationship. Likewise with child relationships. The solution to replace DLRS will need strong ties into an easy-to-use invocable method, and sadly SObjectField
is not yet supported as an argument type for Invocable Apex actions.
Additionally, a note on design -- after even the briefest of forays into the DLRS codebase had me clawing at my eyes, I made the decision to only use one class (and a test class). Isolating the mechanics of the rollups solely within the Rollup
class would mean striking a delicate balance between creating a "god class" and adhering to the Single Responsibility Principle.
Shortcomings aside, the core code in Rollup
is worth examining -- let's go deeper.
Without further ado, let's look at some of the key methods in the initial version of Rollup
:
1link// in Rollup
2link// AVG, MAX, MIN, COUNT operations not yet implemented
3link// because SUM was the only initial ask
4linkprivate enum Op {
5link SUM,
6link UPDATE_SUM,
7link DELETE_SUM
8link}
9link
10link// for really powerful up-front filtering of which items
11link// are rolled up, supplying an implementation of the Evaluator
12link// interface does the trick nicely
13linkpublic interface Evaluator {
14link Boolean matches(Object calcItem);
15link}
16link
17link// refer to the "IsFlaggedBySalesUser" example above
18linkprivate static List<SObject> filter(List<SObject> calcItems, Evaluator eval) {
19link List<SObject> applicableItems = new List<SObject>();
20link for(SObject calcItem : calcItems) {
21link if(eval != null && eval.matches(calcItem)) {
22link applicableItems.add(calcItem);
23link }
24link }
25link return applicableItems;
26link}
27link
28link// key the SObjects passed in to the String value
29link// matching the key on the object where the rollup
30link// will be performed
31linkprivate Map<String, List<SObject>> getCalcItemsByLookupField() {
32link Map<String, List<SObject>> lookupFieldToCalcItems = new Map<String, List<SObject>>();
33link for(SObject calcItem : this.calcItems) {
34link String key = (String)calcItem.get(this.lookupFieldOnCalcItem);
35link if(lookupFieldToCalcItems.containsKey(key) == false) {
36link lookupFieldToCalcItems.put(key, new List<SObject>{ calcItem };
37link } else {
38link lookupFieldToCalcItems.get(key).add(calcItem);
39link }
40link }
41link return lookupFieldToCalcItems;
42link}
43link
44link// your garden-variety dynamic SOQL
45linkprivate List<SObject> getLookupItems(Set<String> objIds) {
46link String queryString =
47link 'SELECT Id, ' + this.lookupObjOpField.getDescribe().getName() +
48link '\nFROM ' + this.lookupObj.getDescribe().getName() +
49link '\nWHERE ' + this.lookupField.getDescribe().getName() + ' = :objIds';
50link return Database.query(queryString);
51link}
52link
53link// the meat of the Rollup
54linkprivate void performRollup(Map<String, List<SObject>> calcItemsByLookupField, List<SObject> lookupItems) {
55link List<SObject> toUpdate = new List<SObject>();
56link for(SObject lookupRecord : lookupItems) {
57link String key = (String)lookupRecord.get(this.lookupField);
58link if(calcItemsByLookupField.containsKey(key) == false) {
59link continue;
60link }
61link
62link List<SObject> calcItems = calcItemsByLookupField.get(key);
63link Object priorVal = lookupRecord.get(this.lookupObjOpField);
64link Object newVal = this.getRollupVal(calcItems, priorVal);
65link lookupRecord.put(this.lookupObjOpField, newVal);
66link toUpdate.add(lookupRecord);
67link }
68link
69link update toUpdate;
70link}
71link
72link// right now, this "works" - but we'll need to further generalize
73link// to support other kinds of rollup operations
74linkprivate Object getRollupVal(List<SObject> calcItems, Object priorVal) {
75link Decimal returnVal = priorVal == null ? 0 : (Decimal)priorVal;
76link for(SObject calcItem : calcItems) {
77link switch on this.op {
78link when SUM {
79link returnVal += (Decimal)calcItem.get(this.opField);
80link }
81link when DELETE_SUM {
82link returnVal -= (Decimal)calcItem.get(this.opField);
83link }
84link when UPDATE_SUM {
85link Decimal oldVal = (Decimal)this.oldCalcItems.get(calcItem.Id).get(this.opField);
86link Decimal newVal = (Decimal)calcItem.get(this.opField);
87link returnVal += (newVal - oldVal); // could be negative, could be positive
88link }
89link when else {
90link throw new IllegalArgumentException('Other rollup op: ' + this.op.name() + ' not yet implemented');
91link }
92link }
93link }
94link return returnVal;
95link}
The framework for rolling values up appears fairly quickly; this isn't really even that many lines of code. Indeed, the rest of the Rollup
is largely defined by constructors at this point, and static methods exposing the rollup operations.
First things first -- that getRollupVal
method needs to decouple itself from the type of rollup being performed. This is a great use-case for inner classes:
1linkprivate Object getRollupVal(RollupCalculator calc, List<SObject> calcItems, Object priorVal) {
2link Rollup rollup = this.getRollupType(priorVal);
3link for (SObject calcItem : calcItems) {
4link rollup.performRollup(calc.op, priorVal, calcItem, calc.oldCalcItems, calc.opField.getDescribe().getName());
5link }
6link return rollup.getReturnValue();
7link}
8link
9linkprivate Rollup getRollupType(Object priorVal) {
10link // fun fact - integers, doubles, longs, and decimals
11link // will ALL return true here
12link if (priorVal instanceof Decimal) {
13link return new DecimalRollup(priorVal);
14link } else {
15link throw new IllegalArgumentException('Rollup operation not defined for: ' + JSON.serialize(priorVal));
16link }
17link}
18link
19linkprivate abstract class Rollup {
20link protected Object returnVal;
21link public Rollup(Object returnVal) {
22link this.returnVal = returnVal;
23link }
24link public Object getReturnValue() {
25link return returnVal;
26link }
27link public abstract void performRollup(Op op, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, String operationField);
28link}
29link
30linkprivate class DecimalRollup extends Rollup {
31link public DecimalRollup(Object priorVal) {
32link super(priorVal == null ? 0 : priorVal);
33link }
34link
35link public override void performRollup(Op operation, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, String operationField) {
36link Decimal returnVal = (Decimal) this.returnVal;
37link switch on operation {
38link when SUM {
39link returnVal += (Decimal) calcItem.get(operationField);
40link }
41link when DELETE_SUM {
42link returnVal -= (Decimal) calcItem.get(operationField);
43link }
44link when UPDATE_SUM {
45link Decimal oldVal = (Decimal) oldCalcItems.get(calcItem.Id).get(operationField);
46link Decimal newVal = (Decimal) calcItem.get(operationField);
47link returnVal += (newVal - oldVal); // could be negative, could be positive
48link }
49link when else {
50link throw new IllegalArgumentException('Other rollup op: ' + operation.name() + ' not yet implemented');
51link }
52link }
53link }
54link}
If you're looking at getRollupType
in the above example and thinking that it looks like the Factory pattern, that's a bingo! Now the type of rollup being performed has been decoupled from the logic necessary to perform the actual rollup. That makes it easy to add in:
Op
enum (currently just SUM, but we'll expand on that shortly)A potential code smell (perhaps obscured by only having one rollup operation defined) is the switch
statement in DecimalRollup
. This piece of logic will have to be replicated in each Rollup
inner class prior to us being able to proceed. However, with a slight shift in perspective comes an object-oriented opportunity to reduce the boilerplate necessary to introduce new rollup types into the mix:
1link// in Rollup.cls
2linkprivate enum Op {
3link SUM,
4link UPDATE_SUM,
5link DELETE_SUM,
6link COUNT, // our first new operation!
7link UPDATE_COUNT,
8link DELETE_COUNT
9link}
10link
11linkprivate Rollup getRollupType(Object priorVal, Op operationType) {
12link // have to use the fully qualified Op name here (including the outer class)
13link // since its type is shadowed in this method
14link if(operationType.name().contains(RollupCalculator.Op.COUNT.name())) {
15link return new CountRollup(priorVal);
16link } else if (priorVal instanceof Decimal) {
17link return new DecimalRollup(priorVal);
18link } else {
19link throw new IllegalArgumentException('Rollup operation not defined for: ' + JSON.serialize(priorVal));
20link }
21link}
22link
23linkprivate abstract class Rollup {
24link protected Object returnVal;
25link public Rollup(Object returnVal) {
26link this.returnVal = returnVal;
27link }
28link // we make this virtual to deal with downcasting
29link public virtual Object getReturnValue() {
30link return returnVal;
31link }
32link public abstract void performRollup(Op op, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, SObjectField operationField);
33link}
34link
35linkprivate virtual class DecimalRollup extends Rollup {
36link public DecimalRollup(Object priorVal) {
37link // much as it pains me to duplicate the null check, it must be done;
38link // we can't reference instance methods till after the super() call
39link super(priorVal == null ? 0 : priorVal);
40link }
41link
42link protected Decimal getDecimalOrDefault(Object potentiallyUnitializedDecimal) {
43link return (Decimal) (potentiallyUnitializedDecimal == null ? 0 : potentiallyUnitializedDecimal);
44link }
45link
46link protected virtual Decimal getNumericValue(SObject calcItem, SObjectField operationField) {
47link return this.getDecimalOrDefault(calcItem.get(operationField));
48link }
49link
50link protected virtual Decimal getNumericChangedValue(SObject calcItem, SObjectfield operationField, Map<Id, SObject> oldCalcItems) {
51link Decimal newVal = this.getNumericValue(calcItem, operationField);
52link Decimal oldVal = this.getNumericValue(oldCalcItems.get(calcItem.Id), operationField);
53link // could be negative, could be positive ... could be 0!
54link return newVal - oldVal;
55link }
56link
57link public override void performRollup(Op operation, Object priorVal, SObject calcItem, Map<Id, SObject> oldCalcItems, SObjectField operationField) {
58link Decimal returnVal = (Decimal) this.returnVal;
59link switch on operation {
60link when SUM, COUNT {
61link returnVal += this.getNumericValue(calcItem, operationField);
62link }
63link when DELETE_SUM, DELETE_COUNT {
64link returnVal -= this.getNumericValue(calcItem, operationField);
65link }
66link when UPDATE_SUM, UPDATE_COUNT {
67link returnVal += this.getNumericChangedValue(calcItem, operationField, oldCalcItems);
68link }
69link when else {
70link throw new IllegalArgumentException('Other rollup op: ' + operation.name() + ' not yet implemented');
71link }
72link }
73link }
74link}
75link
76linkprivate class CountRollup extends DecimalRollup {
77link public CountRollup(Object priorVal) {
78link super(priorVal);
79link }
80link
81link public override Object getReturnValue() {
82link return (Integer) this.returnVal;
83link }
84link
85link protected override Decimal getNumericValue(SObject calcItem, SObjectField operationField) {
86link Decimal potentialReturnValue = super.getNumericValue(calcItem, operationField);
87link return this.getCountValue(potentialReturnValue);
88link }
89link
90link protected override Decimal getNumericChangedValue(SObject calcItem, SObjectField operationField,
91link Map<Id, SObject> oldCalcItems) {
92link Decimal potentialReturnValue = super.getNumericChangedValue(calcItem, operationField, oldCalcItems);
93link return this.getCountValue(potentialReturnValue);
94link }
95link
96link private Decimal getCountValue(Decimal potentialReturnValue) {
97link return potentialReturnValue > 0 ? 1 : potentialReturnValue;
98link }
99link}
Et voila -- in just a few short lines and an additional check in the factory method, we've added a completely different rollup type into the mix. The new CountRollup
class has an interesting condition to it (consistent with the rules for how the COUNT
function works in SOQL) -- it still requires there to be a non-null value on the field within the calculation item that it's comparing to. It may be the case that people simply want to count all child/related objects and roll up their presence to a parent/related object. I'll be curious to hear from you as to whether or not that functionality is desired, prior to implementing a so-called BlindCount
version.
As I added more functionality, it occurred to me that either by wanting to perform many rollups from within a single trigger, or by having rollup operations involving large amounts of records, it might be possible to creep towards the synchronous DML limit of 10,000 DML rows in a single transaction. Making Rollup
perform the bulk of the work async also led to implementing the Data Processor pattern -- by burning a few SOQL queries on the synchronous side, Rollup
can automatically (or through the use of RollupLimit__mdt
custom metadata) scale from running as a Queueable to a Batchable as the size of rollup operations grow. This is the power of elastic scaling!
Today is December 29th. It's not that I took a break from writing this article; I didn't. What happened? Over the hours (and then days) that followed, I began to add additional operations for Rollup
. My assumption -- with the SUM operations for numbers basically "complete" -- surrounding the ease of implementing the rest of the rollup functions slowly began to wither on the vine. I laughed on Christmas when a stranger submitted a PR on one of my open-source repos -- one thing I was no stranger to was putting in long hours on passion projects, and somebody else was clearly taking advantage of the holidays.
What began as a casual foray into adding COUNT
quickly began to escalate outwards as the code smell that I mentioned earlier -- the switch
statement in DecimalRollup
became bigger and bigger.
"This isn't really even that many lines of code" - a younger, more naive version of me (earlier in this article), before the
switch
statement inDecimalRollup
ended up as 50% of the size of the entireRollup
class when I started
The addition of an Invocable entry point for Flows and Process Builders contributed to the rising tide of lines of code -- and at the end of that journey, without many of the rollup functions even implemented yet, I realized it was past time ... the tests needed to be ported over, and new ones created ASAP to ensure what I had so far was going to work.
If you've read The Joys Of Apex before, you know that keeping your Salesforce tests running quickly is something I'm passionate about. This was doubly important in the current context: the tests I'd originally written were tightly coupled to two things that weren't going to make the cut:
One of the reasons that DLRS is so big as a codebase is because it imports a ton of code from FFLib to perform the metadata deployments necessary to create DLRS triggers/rollups as needed. It occurred to me that I was trying to encourage something more along the lines of adding an analytics tag to a website than the sort of service DLRS offerred: I've worked on several analytics implementations, and vetted many vendors. I've never heard of somebody offering a service that didn't require the installation of a JavaScript / backend SDK in order to work (even Cloudflare and other reverse proxy services require up-front configuration). In other words, as I considered how I wanted my tests to work, I had to also solidify the concepts (like Rollup
being more akin to installing Google Analytics) necessary for Rollup
to be used.
Since my prior tests were a no-go, I found myself recalling the immortal words of Kent Beck in "Test Driven Development By Example":
You will often be implementing TDD in code that doesn't have adequate tests. When you don't have enough tests, you are bound to come across refactorings that aren't supported by tests ... what do you do? Write the tests you wish you had.
In order to make the tests run fast, I needed to limit the amount of DML performed. Crucial to that effort would be the following class and seam:
1link// in Rollup.cls
2link@testVisible
3linkprivate virtual class DMLHelper {
4link public virtual void doUpdate(List<SObject> recordsToUpdate) {
5link update recordsToUpdate;
6link }
7link}
8link
9link/**
10link * receiving an interface/subclass from a property get/set (from the book "The Art Of Unit Testing") is an old technique;
11link * useful in limited contexts to get around the classic approach to dependency injection
12link * (such as in this case, when constructor-based DI isn't possible).
13link * It's more palatable in Apex than in many other languages, as a matter of fact -
14link * this is because the @testVisible annotation enforces for us the override only being possible while testing
15link */
16link@testVisible
17linkprivate static DMLHelper DML {
18link get {
19link if (DML == null) {
20link DML = new DMLHelper();
21link }
22link return DML;
23link }
24link set;
25link}
26link
27link// and then in the tests:
28link
29linkprivate class DMLMock extends Rollup.DMLHelper {
30link public List<SObject> Records = new List<SObject>();
31link public override void doUpdate(List<SObject> recordsToUpdate) {
32link this.Records = recordsToUpdate;
33link }
34link}
Wait a minute. Doesn't this look familiar? It should. But because I was intent on keeping everything within one class, I couldn't import dependencies -- I either needed to recreate them inside of the Rollup
class, or do without them. The Factory pattern for dependency injection and the Repository pattern for strongly-typed and easily mocked queries were both discarded as a result. That left me with the DML Mock pattern -- and sure enough, all of the tests are lightning-fast as a result. Here's a simple one:
1link// in RollupTests.cls
2link@isTest
3linkstatic void shouldSumFromTriggerAfterInsert() {
4link DMLMock mock = getMock(new List<Opportunity>{ new Opportunity(Amount = 25), new Opportunity(Amount = 25) });
5link Rollup.triggerContext = TriggerOperation.AFTER_INSERT;
6link
7link Rollup rollup = Rollup.sumFromTrigger(
8link Opportunity.Amount,
9link Opportunity.AccountId,
10link Account.Id,
11link Account.AnnualRevenue,
12link Account.SObjectType
13link );
14link
15link System.assertEquals(true, mock.Records.isEmpty());
16link
17link Test.startTest();
18link rollup.runCalc();
19link Test.stopTest();
20link
21link System.assertEquals(1, mock.Records.size(), 'Records should have been populated SUM AFTER_INSERT');
22link Account updatedAcc = (Account) mock.Records[0];
23link System.assertEquals(50, updatedAcc.AnnualRevenue, 'SUM AFTER_INSERT should add the original opportunity amount');
24link}
One hidden side-effect is the tying of the Opportunities to the Account in question - that happens in the getMock
method:
1link// in RollupTests.cls
2linkprivate static DMLMock getMock(List<SObject> records) {
3link Account acc = [SELECT Id FROM Account];
4link for (SObject record : records) {
5link record.put('AccountId', acc.Id);
6link }
7link
8link return loadMock(records);
9link}
10link
11link// ...
12link
13linkprivate static DMLMock loadMock(List<SObject> records) {
14link Rollup.records = records;
15link Rollup.shouldRun = true;
16link DMLMock mock = new DMLMock();
17link Rollup.DML = mock;
18link
19link return mock;
20link}
There are, in fact, only three other helper methods (non test methods) in the entire test class. Once I had tests that covered the basics of what I'd written for the SUM
and COUNT
implementations, it was time to approach this thing TDD-style:
This simple rhythm helped me to immediately spot a flaw in the code for the Invocable method as I worked to create tests for this new functionality -- it was being fed into a method that used the current Trigger context (the TriggerOperation
enum) to figure out what kind of rollup operation was being performed. That's also where the Rollup.records
variable came from in the loadMock
code, above; a way to stub in the trigger records without actually having to require a trigger being run.
When you look at the first ~20 lines of Rollup
, you can see the use of these @testVisible
private static variables as the "poor man's dependency injection:"
1link /**
2link * Test override / bookkeeping section. Normally I would do this through dependency injection,
3link * but this keeps things much simpler
4link */
5link @testVisible
6link private static Boolean shouldRun;
7link @testVisible
8link private static Boolean shouldRunAsBatch = false;
9link @testVisible
10link private static TriggerOperation triggerContext = Trigger.operationType;
11link @testVisible
12link private static Map<Id, SObject> oldRecordsMap;
13link @testVisible
14link private static List<Rollup__mdt> rollupMetadata;
15link @testVisible
16link private static List<SObject> queryRecords;
17link @testVisible
18link private static RollupLimit__mdt defaultRollupLimit;
19link @testVisible
20link private static RollupLimit__mdt specificRollupLimit;
There are some juicy hints, above, of what was ultimately to come.
Adding the CMDT-record driven rollups was simple now that I had a burgeoning test suite and two different possible points of entry (Invocable / Trigger-based) into Rollup
. Indeed, because the test suite was expansive and I believe that CMDT rollups form the core of peoples' rollup needs, refactoring the code to support custom metadata as a first-class citizen became (truly) a joy. You can tell that the code is really built around it because there are only 3 lines of code in the public-facing method:
1link// in Rollup.cls - don't mind that null argument below, it's for the custom Evaluator interface
2linkpublic static void runFromTrigger() {
3link SObjectType sObjectType = getTriggerRecords().getSObjectType();
4link List<Rollup__mdt> rollupMetadata = getTriggerRollupMetadata(sObjectType);
5link runFromTrigger(rollupMetadata, null).runCalc();
6link}
This is the "Google Analytics" approach: in order to use Rollup
, all you need to do is add one line of code to your triggers:
1linkRollup.runFromTrigger();
Unless you need a ton of customization, it's really as simple as that. Please note: this requires your trigger to use the following contexts: after insert, after update, and before delete. Without those in place, Rollup
will not function as designed for trigger-based rollups!
The rest of the info -- about which rollups need to be processed for the trigger in question -- can all live in the Rollup__mdt
Custom Metadata:
Calc Item
Calc Item
Lookup Field On Calc Item
on your lookup object. This is a Field Definition field; you can only select from the list of available fields after having made an object-level selection for Lookup Object
Lookup Object
There are some peculiarities within Apex when working with Entity Definition and Field Definition-based Custom Metadata fields, which I will detail just below in the key takeaways section!
This is also the preferred method for scheduling; while I do expose the option to schedule a rollup from Apex, I find the ease of use in creating Scheduled Flows in conjunction with the deep power of properly configured Invocables to be much more scalable than the "Scheduled Jobs" of old. This also gives you the chance to do some truly crazy rollups -- be it from a Scheduled Flow, an Autolaunched Flow, or a Platform Event-Triggered Flow. As long as you can manipulate data to correspond to the shape of an existing SObject's fields, they don't even have to exist; you could have an Autolaunched flow rolling up records when invoked from a REST API so long as the data you're consuming contains a String/Id matching something on the "parent" rollup object.
There were quite a few learning moments as I worked through rollup edge cases; I've chosen to spend the rest of this article articulating ones that I think might be helpful or interesting to you in your own Salesforce journey:
There's not a whole lot of documentation out there about Entity / Field Definition-basd CMDT. They are as good as they sound -- giving users of your CMDT object-level and field-level safety when letting them select fields, but something interesting that I found while working with Field Definition fields in Apex is that they are stored as "ObjectName.FieldName" in the database. This roughly corresponds to a string-level representation of what an SObjectField
type is written as:
1link// if we query for the Rollup__mdt shown earlier
2linkRollup__mdt rollupMetadata = [SELECT RollupFieldOnLookupObject__c FROM Rollup__mdt LIMIT 1];
3linkSystem.debug(rollupMetadata.RollupFieldOnLookupObject__c); // outputs "Opportunity.Amount", for example
4linkSObjectField opportunityAmount = Opportunity.Amount;
5linkSystem.debug(opportunityAmount); // ouputs ... "Amount" ... so you know that somebody overrode the "toString()" method for this class!
When working with dynamic fields in SOQL, developers frequently use the DescribeSObjectResult
and DescribeFieldResult
classes that give you access to metadata about objects/fields ... but in this case, I found I had to create a helper method specifically for working with the String-based version of the Field Definition values coming in from the Rollup__mdt
records:
1link// takes a string from CMDT like "Opportunity.Amount" and returns just the field name: "Amount"
2link// this allows us to match the String-based version of the field with its corresponding SObjectField
3link// by calling describeForSObject.fields.getMap().get(theFieldNameReturnedFromgetParedFieldName)
4linkprivate static String getParedFieldName(String fullFieldName, DescribeSObjectResult describeForSObject) {
5link return String.isBlank(fullFieldName) ? '' : fullFieldName.replace(describeForSObject.getName() + '.', '');
6link}
You might remember from the Apex Enum Class Gotchas article that sending the name()
value for an enum is the only way to properly deserialize the enum when you're ingesting data either in Apex or in another service. That's all well and good -- but, as it turns out, you can't deserialize directly to the String-based enum:
1link// in Rollup.cls - making this a public enum doesn't change the result
2linkprivate enum Op {
3link SUM,
4link UPDATE_SUM,
5link DELETE_SUM
6link // etc
7link}
8link
9linkRollup__mdt rollupMetadata = methodWhereWeGetTheMetadata();
10link// this is a crazy thing and I wouldn't have wanted to do it anyway, but science ...
11link// if you ACTUALLY need to do these things from within Apex, I highly recommend the use of the JSONGenerator class
12linkString operationFromMetadata = '{ "op" : "' + rollupMetadata.RollupType__c + '"}';
13linkOp operation = (Op)JSON.deserialize(operationFromMetadata, Op.class);
14linkSystem.debug(operation);
15link// outputs null
16link
17link// you COULD make a wrapper class
18linkprivate class OpWrapper {
19link public Op op { get; set; }
20link}
21linkOpWrapper opWrapper = (OpWrapper)JSON.deserialize(operationFromMetadata, OpWrapper.class);
22linkSystem.debug(opWrapper);
23link// outputs: "OpWrapper:[Op=SUM]", for example
Keying the string from the CMDT to be valid JSON and creating a wrapper class left me feeling a little ill, so instead I went with a lazily-loaded Map<String, Op>
using the included values()
method present on all enums:
1linkprivate static Map<String, Op> opNameToOp {
2link get {
3link if (opNameToOp == null) {
4link opNameToOp = new Map<String, Op>();
5link for (Op operation : Op.values()) {
6link opNameToOp.put(operation.name(), operation);
7link }
8link }
9link return opNameToOp;
10link }
11link set;
12link}
It's not perfect, but it's better than crafting JSON to get around not being able to cast from a String to an enum, and not being able to reference the enum by its name()
in any other way.
This is probably true of other field types (honestly, at this point, nothing would really surprise me), but it came up frequently for me while testing with Date
/ Datetime
fields, as well as multi-select picklists (the Devil's picklists, some would say).
It did come as a real surprise to me as I was writing tests; I had used only the Account and Opportunity objects so far in my tests, in an effort to keep them as generic as possible. I wanted to write a test using a Date
field that wasn't required on insert to test my DefaultFieldInitializer
(more on that in a second). While you can MIN
or MAX
the Opportunity's CloseDate
field, the "parent" object I had been using as the target of my rollups, Account, didn't have a Date
field on it. In retrospect, I could have used the Contract object (and later I would), but I'm glad I didn't, as I might not have encountered this charming error message otherwise: System.QueryException: There's a problem with your query: field ActivityDate does not support aggregate operator MAX
. Uhhh, OK. Thanks for that, Salesforce.
Wherever possible, I've tried to make the code resilient to the idiosyncracies of the platform; for operations like this, if that means doing the damn MIN
/ MAX
myself, so be it:
1link// MIN/MAX is allowed, but not for all fields, and not consistently. Go figure!
2linkprotected virtual override Object calculateNewAggregateValue(Set<Id> excludedItems, Op operation, SObjectField operationField, SObjectType sObjectType) {
3link Object aggregate;
4link try {
5link aggregate = super.calculateNewAggregateValue(excludedItems, operation, operationField, sObjectType);
6link } catch (Exception ex) {
7link // technically a System.QueryException, but I figure we might as well catch em all and try like hell to aggregate anyway
8link Decimal minOrMax;
9link List<SObject> allOtherItems = Database.query('SELECT ' + operationField + ' FROM ' + sObjectType + ' WHERE Id != :excludedItems');
10link for (SObject otherItem : allOtherItems) {
11link Decimal otherItemDate = this.getDecimalOrDefault(otherItem.get(operationField));
12link if (otherItemDate != null && operation.name().contains(Op.MAX.name()) && (minOrMax == null || otherItemDate > minOrMax)) {
13link minOrMax = otherItemDate;
14link } else if (otherItemDate != null && operation.name().contains(Op.MIN.name()) && (minOrMax == null || otherItemDate < minOrMax)) {
15link minOrMax = otherItemDate;
16link }
17link }
18link if (minOrMax == null) {
19link aggregate = operation.name().contains(Op.MIN.name()) ? FieldInitializer.maximumLongValue : FieldInitializer.minimumLongValue;
20link } else {
21link aggregate = minOrMax;
22link }
23link }
24link
25link return aggregate;
26link}
27link// not pictured - the "hot as hell" section where min/max gets tabulated for multi-select picklists. Yowza! Feel the burn!
I thought I'd be able to simply query a given field on the rollup object in question, test for its type using instanceof
, and then move on to rolling up. Unfortunately, that approach failed on my very first test where the rollup object didn't have the rollup field initialized. I needed to find something -- anything -- that would give me a clue about what the type was for a given field at runtime, and found it in the DisplayType
enum. Indeed, I found out after creating the default value initializer that this was the same method employed by a number of dynamic Apex test libraries:
1link// in Rollup.cls
2linkprivate virtual class DefaultSObjectFieldInitializer {
3link public final Datetime defaultDateTime = Datetime.newInstanceGmt(1970, 1, 1);
4link public final Long maximumLongValue = (Math.pow(2, 63) - 1).longValue();
5link public final Long minimumLongValue = this.maximumLongValue * -1;
6link
7link public virtual Object getDefaultValue(SObjectField field) {
8link DescribeFieldResult fieldDescribe = field.getDescribe();
9link if (fieldDescribe.isDefaultedOnCreate()) {
10link return fieldDescribe.getDefaultValue();
11link }
12link // not surprisingly, "getDefaultValue" on the DescribeFieldResult returns null for fields without default values
13link // this is a shame - all types *should* have default values. Instead, we have the privilege of getting to initialize them
14link Object initializedDefault;
15link switch on fieldDescribe.getType() {
16link when CURRENCY, DOUBLE, INTEGER, LONG, PERCENT {
17link initializedDefault = 0;
18link }
19link when DATETIME {
20link initializedDefault = this.defaultDateTime;
21link }
22link when DATE {
23link initializedDefault = this.defaultDateTime.dateGmt();
24link }
25link when TIME {
26link initializedDefault = this.defaultDateTime.timeGmt();
27link }
28link when STRING, ID, TEXTAREA, URL, PHONE, EMAIL{
29link initializedDefault = '';
30link }
31link when PICKLIST, MULTIPICKLIST {
32link // more on this part in a second
33link initializedDefault = new PicklistController(field.getDescibe()).getDefaultValue(field);
34link }
35link when else {
36link throw new IllegalArgumentException('Field: ' + field + ' of type: ' + fieldType.name() + ' specified invalid for rollup operation');
37link }
38link }
39link return initializedDefault;
40link }
41link}
In the end, this Anonymous Apex script proved invaluable for hunting down standardly available fields of special types like TIME
:
1linkpublic static void printDisplayTypeInfo(DisplayType desiredType) {
2link Map<String, SObjectType> namesToTypes = Schema.getGlobalDescribe();
3link for(SObjectType sType : namesToTypes.values()) {
4link Map<String, SObjectField> fields = sType.getDescribe().fields.getMap();
5link for(String fieldName : fields.keyset()){
6link SObjectField field = fields.get(fieldName);
7link DescribeFieldResult describeResult = field.getDescribe();
8link if(describeResult.getType() == desiredType && describeResult.isUpdateable()) {
9link System.debug('SObjectType: ' + sType);
10link System.debug(describeResult.getName());
11link // you could put a return statement here to only print the first result found
12link }
13link }
14link }
15link}
I had never worked with the ContactPointAddress or ContactPointEmail objects before; they're relatively new additions to the system, and it was fun to learn a bit more about them while using them to wire up different zany relationships.
I've talked quite a bit about the infamous switch
statement now in the DecimalRollupCalculator
. After implementing all of the rollup operations for numbers, I teetered at the precipice -- how to take the logic in DecimalRollupCalculator
and generalize it so that other subclasses could make use of it. As I considered the horrors of a switch
statement that needed to differentiate not only between different rollup operations, but also their context (was it an insert? an update? a delete??), my stomach began to twist. I had just written my first failing test for implementing MAX for Datetimes. Would my response really be to copy pasta?
Then it hit me. Datetimes are all stored uniformly within Salesforce in UTC time. UTC can be represented by numbers. Dates
are just Datetime
s with a zero'd out Time
section. The game was on. In the end, this section truly took very little additional code to get right:
1link// omitting the Datetime parent class, which itself descends from DecimalRollupCalculator
2link// the reason should be clear if you take a peek at the source code. While it's simple, an
3link// excerpt from DatetimeRollupCalculator is shown below in the "SOQL Drops Milliseconds From Datetimes ..." section
4linkprivate class DateRollupCalculator extends DatetimeRollupCalculator {
5link // for Date, it's not necessary to override the "getDecimalOrDefault" method in DatetimeRollupCalculator
6link // because the conversion only happens in "getReturnValue"
7link public DateRollupCalculator(Object priorVal, SObjectField operationField) {
8link super(Datetime.newInstanceGmt((Date) priorVal, Time.newInstance(0, 0, 0, 0)), operationField);
9link }
10link
11link public override Object getReturnValue() {
12link return ((Datetime) super.getReturnValue()).dateGmt();
13link }
14link }
15link
16link private class TimeRollupCalculator extends DatetimeRollupCalculator {
17link public TimeRollupCalculator(Object priorVal, SObjectField operationField) {
18link super(Datetime.newInstanceGmt(FieldInitializer.defaultDateTime.dateGmt(), (Time) priorVal), operationField);
19link }
20link
21link public override Object getReturnValue() {
22link return ((Datetime) super.getReturnValue()).timeGmt();
23link }
24link
25link protected override Decimal getDecimalOrDefault(Object potentiallyUnitializedDecimal) {
26link Datetime defaultDatetime;
27link if (potentiallyUnitializedDecimal instanceof Time) {
28link defaultDatetime = Datetime.newInstanceGmt(FieldInitializer.defaultDateTime.dateGmt(), (Time) potentiallyUnitializedDecimal);
29link } else if (potentiallyUnitializedDecimal instanceof Decimal) {
30link defaultDatetime = Datetime.newInstance(((Decimal) potentiallyUnitializedDecimal).longValue());
31link } else {
32link defaultDatetime = FieldInitializer.defaultDateTime;
33link }
34link return defaultDatetime.getTime();
35link }
36link }
That commit -- and the powerful realization that I'd been able to add a possible sixty-three different permutations for rollup operations in what amounted to a mere ~32 lines of code ... that's the power of Object-Oriented Programming!
Edit -- what I failed to mention, upon originally publishing this article, was that there was a duplicated switch statement (for String-based rollups); a detail that had bothered me in the days leading up to release, but which I didn't have the time (or energy, after several marathon days spent "finishing" Rollup
) to address prior to launching. Several days after releasing, I went back and made use of the Chain Of Responsibility to break up this duplication. As is common when virtual methods are added at the parent level, this actually ended up increasing the lines of code in the final Rollup
file. At the same time, indentation (one of the great sins present in the use of switch statements) was much reduced. What was left? One switch statement to rule them all, with virtual methods to bind them. While lines of code can be a metric for complexity, it also often ends up being deceptive -- the code reads more like a story, now, where rollup operations are explicitly opted into, instead of having to scan through a switch statement to determine which operation leads to which result.
While working on MAX/MIN based code, I realized quickly that there were only two reliable sentinel values when it came to numbers -- the maximum possible number that would fit into a 64-bit number (a Long
), and the minimum possible number. What bit me really, really hard while working on implementing min/max for picklists? To anwer that, first an aside -- min/maxing on a picklist is supposed to return the deepest possible entry in the picklist (for MAX), or the closest to the top of the picklist (for MIN). Picklists have many ... interesting ... subleties in Salesforce, and the implicit concept of "rank" (just look at the Path components for Lead Status or Opportunity Stage Name) is just one of their many quirks.
If a value doesn't exist on the lookup object, that value should always lose; on a MIN it should be greater than the "rank" of any other field (so that any comparison to it leads to a truthy "less than" evaluation); on a MAX it should be less than the rank of any other field (likewise; it should lead to a truthy "greater than" evaluation). Making an object to perform these evaluations for picklists was a great exercise in Object Oriented Programming -- frequently throughout this project, that proved to be the case. Making the PicklistController
inner class descend from the DefaultFieldInitializer
made sense in the context of the object hierarchy. Though the PicklistController
had many more responsibilities, it could also handle setting the default value for a picklist with ease.
Where things finally took a turn for the worse was when my first picklist test for ensuring MIN/MAX was working correctly failed. The logic looked perfect -- what could be the issue? It took me a painful 30 minutes to see the issue, so deeply ingrained were my assumptions about the way that Long
values would translate to Integers
. Only at the last second did I truly comprehend that I had betrayed myself:
1linkprivate class PicklistController extends DefaultSObjectFieldInitializer {
2link// etc ...
3link
4link private Integer getSentinelValue(Boolean isMin) {
5link return (isMin ? this.maximumLongValue : this.minimumLongValue).intValue();
6link }
7link}
Passing Boolean
values as method arguments is always to be strongly discouraged, but this method call was already on the tail end of a ternary and I felt, when creating it, that I had no other option. I will say, though, that I once had a coworker who was deeply passionate about using nested ternaries. I lost track of him over the years, but ... perhaps the issue would have been clearer to me if the culprit had been in a truly doubled ternary; after all, who in their right minds could avoid investigating such a travesty?
In any case. Do you know what the integer values are for the minimum and maximum Long
values (-2^63 and 2^63, respectively)?
1linkLong maximumLongValue = (Math.pow(2, 63) - 1).longValue();
2linkLong minimumLongValue = maximumLongValue * -1;
3link
4linkSystem.debug(maximumLongValue); // prints: 9223372036854775807
5linkSystem.debug(minimumLongValue); // prints: -9223372036854775807
6link
7link// Now as ints!
8linkSystem.debug(maximumLongValue.intValue()); // prints -1
9linkSystem.debug(minimumLongValue.intValue()); // prints 1
Ouch. Well, that explained my failing test! My understanding (after a brief foray into the details) is that casting to Integer
or calling intValue()
is safe within the bounds of the 32-bit allowed integer sizes, but after that all bets are off. Creating the min/max integer bounds did the trick wonderfully. Running into edge cases like this can hurt, but it also expands the mind -- you only get bit by something like this once before knowing to look out for it next time around. Plus, I ended up being able to get rid of the Boolean
passing altogether -- two birds with one stone!
This was a fun one -- and one that I was already aware of from lurking on the SFXD Discord. This is a real pain -- especially in testing --, but in the end the following bit did the trick nicely:
1link// one of the worst things about SOQL is that Datetimes retrieved have the millisecond values truncated
2linkDatetime datetimeWithMs = potentiallyUnitializedDecimal instanceof Decimal
3link ? Datetime.newInstance(((Decimal) potentiallyUnitializedDecimal).longValue())
4link : ((Datetime) potentiallyUnitializedDecimal);
5link// reading through the source code provides a more cogent rationale
6link// for the above eyesore over anything I can muster here.
7linkreturn Datetime.newInstanceGmt(
8link datetimeWithMs.yearGmt(),
9link datetimeWithMs.monthGmt(),
10link datetimeWithMs.dayGmt(),
11link datetimeWithMs.hourGmt(),
12link datetimeWithMs.minuteGmt(),
13link datetimeWithMs.secondGmt()
14link )
Well, it's out there now. This article -- and the corresponding code -- has consumed an enormous quantity of time since work began on it in earnest in early December. I would highly recommend developers check out the source code (and the travelogue style commit history). Over the coming months I plan to add more functionality to Rollup
-- for now, I'm hopeful that you'll consider trying it out. It's efficient, scales elastically, allows for rollups on fields (like Task.ActivityDate
) that don't always allow for rollups in SOQL, and is well-tested.
I'm aware that Rollup
doesn't hit 100% feature-parity versus DLRS ... and though I have plans to meet that challenge, as well, I believe that we're well past the fabled "80% of the functionality" stage. If your org (like many out there) is struggling under the weight of DLRS' auto-spawned triggers, I'm confident that Rollup
will be a valuable tool for both the declaratively-minded as well as the developers out there.
My original intent was to finish this article by December 27th, the one year anniversary of the Joys Of Apex. Despite some insanely long days spent writing, that didn't quite happen. Despite that, I just wanted to say that the readers of this series have helped ease the burden in a year that was extremely challenging for many people. Stuck inside for large portions of time, I took to writing -- and it shows. More than ten of the articles I wrote over the last year came out during the first 2 months of the pandemic -- some only days apart from one another. I do not advertise, and have not attempted to monetise in any way the incredible surge of traffic my personal website has experienced as a result. My intent is to provide readers with free content and materials to refer back to. Indeed, the only work I've done on the site over the past year was immediately preceeding this article, as I spruced up the Joys Of Apex blog page to better show off the posts.
All of that is to say -- thank you for an incredible year. Here's to hoping that 2021 will prove a better year for the world, and for you.
The original version of Replacing DLRS With Custom Rollup can be read on my blog.
Home Advanced Logging Using Nebula Logger Apex Logging Service Apex Object-Oriented Basics Batchable And Queueable Apex Building A Better Singleton Continuous Integration With SFDX Dependency Injection & Factory Pattern Enum Apex Class Gotchas Extendable Apis Formula Date Issues Future Methods, Callouts & Callbacks Idiomatic Salesforce Apex Introduction & Testing Philosophy Lazy Iterators Lightweight Trigger Handler LWC Composable Modal LWC Composable Pagination LWC Custom Lead Path Mocking DML React Versus Lightning Web Components Refactoring Tips & Tricks Replacing DLRS With Custom Rollup Repository Pattern setTimeout & Implementing Delays Sorting And Performance In Apex Test Driven Development Example Testing Custom Permissions The Tao Of Apex Writing Performant Apex Tests