Data synchronization: Incremental push for composite entities in Dynamics 365 Finance & Operations

July 30 2018

A retail client needed to synchronize sales orders between their Dynamics 365 Finance and Operations system and their retail management solution. Keeping the retail system up to date required the frequent export and import of the entire database of sales orders, which imposed a significant performance cost.

It is quite common for organizations to have to synchronize data between their Dynamics 365 ERP system, and third-party applications. Achieving synchronization by exporting the entire table - a "full push"- from Dynamics 365 to the other application quickly becomes inefficient, and cumbersome as the size of the database and frequency of synchronization increase. A more efficient way to keep databases synchronized is an "incremental push" - syncing only the entities that have been changed, added or deleted.

In Dynamics AX 2012 R3, Microsoft introduced the power of Microsoft SQL Server Change Tracking to track the changes in AX tables. Change Tracking allows an incremental push of only those entities that have changed. Firstly, Change Tracking needs to be turned on for the target entity. An incremental push can then be selected as the default refresh method.

Enable change tracking for the target entities

Change Tracking needs to be enabled for the target entity

Select synchronization method--full or incremental push

Select synchronization method - full or incremental push

However, not all entities can be handled in this way. Incremental push for composite entities is not supported in Dynamics 365 Finance and Operations.

D365 throws an error message for an incremental push with composite entities

Dynamics 365 throws an error when an incremental push of a composite entity is attempted

The issue for the retail client is that they handle their sales orders as composite entities. The Sales Order composite entity is comprised of an individual Sales Order header entity, Sales Order Line entity, and Header and Line Charge entities. A solution was needed to enable Change Tracking of composite entities which would then allow an incremental push.

The solution involved extending the following objects in the Dynamics Data Management Framework:

  • DMFEntityBase
  • DMFEntityWriter
  • DMFQuickImportExportFormHelper

The following new objects were created:

  • DMFQuickImportExportFormHelperEventHandler: Checks whether header entity has Change Tracking enabled and prepares data to write to target.
  • DMFEntityWriterEventHandler: Responsible for writing data to target, such as XML or Excel file, overwriting standard method to accept composite entity.
  • DMFDefinitionGroupExecutionTableEventHandler: Provides the status of any export/import run.

This customization allows Change Tracking to be applied to the header of composite entities. This, in turn, allows the retailer's Dynamics 365 system to synchronize only new or changed sales orders with its retail management system with an incremental push rather than having to full push the whole database. This solution delivers substantial performance benefits.

FREE Membership Required to View Full Content:

Become a MemberLogin
Joining MSDynamicsWorld.com gives you free, unlimited access to news, analysis, white papers, case studies, product brochures, and more, and it’s all FREE. You’ll also have the option to receive periodic email newsletters with the latest relevant articles and content updates. Learn more about us here
About Simon Barrett

Simon Barrett is Business Development Manager for ANEGIS Consulting, and a Licensing Microsoft Certified Professional, in Wroclaw, Poland. He specializes in complex, high-impact deployments.

More about Simon Barrett

Comments

Gibon's picture

Your post is like a drop in the sea :)
I have not found anywhere else that incremental push on composite entities could be posible to use.
Although you wrote three classes that needs to be extensioned can you be more specific which methods?
An example would be appreciated.
Thanks.