Harnessing the Power of Microsoft Dynamics CRM's New Bulk Data Load API

Microsoft Dynamics CRM has never been an information island by itself. It is often the case that you need to integrate your CRM data with other applications or databases in your organization or from third-party partners.

When it comes to Microsoft Dynamics CRM data integration, the performance of data loading has always presented a challenge. To be more specific, writing data through the CRM web service interfaces is slow, just like any other web service interfaces. With a direct database connection, you can potentially write thousands or even millions of records to your target table within a second. However, when you write to CRM web service interfaces, the best you can typically expect would be a few dozen records per second using a single-threaded data load process that targets a single CRM server.

If you are a Microsoft Dynamics CRM Online user, your data load would be even slower, largely due to the poor network performance. This performance has been one of major barriers for enterprise customers to move to the cloud.

Particularly when your CRM server is on the cloud, your data load performance is largely affected by network latency. For instance, if the network latency to your CRM Online server is 90ms (70-100ms is very common, and we have seen 400ms and even more), for every single request you send to CRM online, the message would takes 90ms to travel to the server and receive feedback, but the actual processing time might be just only about 10ms or even less, so in this case we have wasted 90% of processing time waiting for the message to travel on the wire.

In order to address the challenges of large-scale data loads, the Microsoft Dynamics CRM platform now offers a new ...