Loading massive amounts of data
When loading lots and lots of data into the Salesforce Platform, we’re essentially concerned with how we can get as much data as possible into our Salesforce instance, reliably, in the shortest time possible. Let’s imagine we have 20 million records to load. Thinking in terms of serial versus parallel processing, we can view our loading scenario in two ways:
- Loading 20 million records sequentially (serial)
- Loading 20 million records in parallel, by breaking down the 20 million records into smaller batches, inserting them in parallel, and taking less time
To load lots of data quickly, we need to optimize our parallel data loads. There are several steps we can take before loading any data to speed up the load operations. Deferring sharing calculations until after the load operation completes will mean sharing recalculations will be run once rather than on every record batch being processed. Disabling any logic that may...