KingswaySoft's SSIS Integration Toolkit v3.0 to Utilize New Bulk Data API for Microsoft Dynamics CRM
For all the interest in the latest releases for Microsoft Dynamics CRM Online and CRM 2011 On Premise, the new bulk data API may have received less attention than it deserved. But if your job description includes includes import or export of large CRM data sets, the bulk data API has the potential to offer serious performance improvements and enable better throughput.
KingswaySoft, makers of the integration and migration solution SSIS Integration Toolkit for Microsoft Dynamics CRM, have announced that version 3 of their product, which is going to be ready for release at the same time as Update Rollup 12, will be one of the very first data integration solutions in the market to support the new bulk API, both for CRM Online and for Dynamics CRM on premise.
SSIS Integration Toolkit utilizes Microsoft SQL Server Integration Services (SSIS) as the ETL platform and data load engine. The new version of SSIS Integration Toolkit is its third major release.
According Daniel Cai, the principle developer of KingswaySoft and a Microsoft Dynamics CRM MVP, the SSIS Integration Toolkit represents something of a new chapter in his company's quest to find a better way to complete large scale CRM data migration and integration efforts. And the new bulk data API offers a major opportunity to improve performance in version three of the toolkit.
"We see the bulk data API as a great technique to improve data load performance since CRM web service is slow, as any web service interface is slow," says Cai. "Particularly when CRM Online is involved, data load is one of the major barriers for enterprise clients to move to the cloud. For example, I can load about 60 records per second to a CRM on-premise environment, but I can only write about 2-3 records per second to a CRM online environment. We see that the CRM bulk API is a game changer when it comes to CRM online data integration. Also, we recognize that managing CRM optionset value is a challenge. Often there is a need to create a new option value to a CRM optionset/picklist field when we receive data from external applications (such as ERP), so we decided to add an extra component to our toolbox, which is the OptionSet mapping component."
Cai has been involved in several CRM projects with months-long data migration and integration efforts and understands the level of complexity these jobs can have.
"Since 2008 I have been engaged in large-scale Dynamics CRM projects as a developer. We had data migration and integration needs for almost every project," says Cai. "I used to write code from scratch [for data migration and integration needs], and it was an extraordinary amount of work for almost all projects that I have been involved in since the CRM service interface is very complex. It has many intricacies dealing with many special cases (special entities, special fields, special web service messages, etc.). I recognize that this is a challenge for almost everyone in the community, so I decided to write an SSIS adapter to make the job easier. With the availability of SSIS Integration Toolkit, I can comfortably say that I can finish the data migration and integration development in a few weeks."
KingswaySoft released its first version of its SSIS Integration Toolkit on Jan 20, 2012, and it was well received by clients and partners. "We received extraordinary amount of feedback and comments about the toolkit from our clients and partners including some creative ideas, comments and feedback, which gave birth to our second major release on September 25, 2012," Cai explained.
Version 3 has already had a positive response. According to Cai, one of the early adopters of the toolkit, based in Australia, has mentioned that their solution's data load process has become several times faster after using the Bulk API feature when using the toolkit with their CRM online environment.
In addition to the bulk API support, SSIS Integration Toolkit v3.0 for Microsoft Dynamics CRM also provides a few new features, including:
- a new action called ExecuteWorkflow was added to the CRM destination component that allows to execute CRM workflows so that you can use SSIS to schedule CRM workflows which is a capability that is not available natively through the Dynamics CRM platform
- a new SSIS data transformation component which allows to manage the translation and mapping of CRM OptionSet values, including the capability to create new options when no-match is found
- an enhanced Text Lookup feature, which allows you to perform lookup to CRM records using text values against any text fields.
SSIS Integration Toolkit is offered in three different versions.
- Developer edition, which is free to use, and it works within the development environment (Visual Studio) without requiring a commercial license.
- Perpetual license, a commercially licensed version which includes one-year maintenance and support, and can be used anywhere including the development environment or outside the development environment.
- One-year subscription license, a commercially licensed version which includes maintenance and support for the entire subscription period, and can be used anywhere including the development environment or outside the development environment.
In order to support the bulk API, a new parameter has been added to the toolkit's CRM destination component, called BatchSize. Once specified, the adapter will leverage the new CRM bulk API by submitting requests in a batched fashion.
KingswaySoft is a small privately held software company specializing in delivering high quality software solutions using latest Microsoft technologies, with a special focus on Microsoft Dynamics CRM and business intelligence software development.