The Microsoft Dynamics AX world changed with the release of Dynamics 365 for Operations (formerly known as AX 7). The traditional methods of data integration are no longer the only way to approach today's challenges across the supply chain. Dynamics 365 for Operations creates new challenges: the whole integration layer needs to be re-written using a different tool set. But it also creates an opportunity for a level of standardization that we have not seen before.

In this article, I hope to give the reader a better appreciation for the challenges we all will be facing and some of the ways in which businesses will choose to meet them. I believe successfully navigating this transition is crucial to the success of small- to mid-size companies and to Dynamics 365 itself.

What is data integration?

Data integration is the process of communicating with the outside world. Whether it be other ERP systems, warehouse management (WMS) systems, messaging systems, or any other system managing a process that can provide and consume structured information. The primary purpose is to provide information that either 1) initiates a process or 2) reports the results of a process. These integrations vary in complexity, which makes each integration unique in its requirements.

This definition is particularly true when it is used to support automation. The process of establishing automation within a business takes into consideration the devices which send and receive the information, and the transposition of outgoing messages to a format that can be consumed by the receiving entity. The process also needs to provide for information that is needed, but which may not be in the message proper.

We typically call this effort process mapping. This information is then consumed and provides the information needed for the proper execution of the process.

A typical example would be sales orders. The data integration process must support many decisions and requirements that will be unique for each organization. This complex rule set typically must be satisfied in the host system before completing the downstream process.

To really be successful we must have better ways to support these processes efficiently.

History of data integration

Electronic messaging in the manufacturing and distribution space has frequently been associated with a standard called electronic data interchange (EDI). EDI was the industry's answer to creating universal standards of communication generally affiliated with certain industry segments. Automotive, retail, and healthcare would each has its own set of standards, based on its unique informational requirements.

What we must remember is that these standards came from a time in our history (about 30 years ago) when the computer industry had not embraced standardization within their operating environments. In fact, they made it difficult to communicate among other systems. These circumstances created huge gaps and unbelievable complexity. Such things as communication protocols (SYNC, ASYNC, BISYNC) and character encodings (ASCII, EBCDIC) were different between systems. Keep in mind the standards were created even before the emergence of today's internet and standard communications protocols like TCP/IP.

Many of those barriers from 30 years ago, no longer exist; still, the stigmas of high cost and exceptional complexity are believed by most people. We used to say "the quickest way to get fired is to be assigned to the EDI project."

Today, the ability to send and receive information along with the ability to interpret the information is in a much better place; still, many environments exist outside the standards. The good news is that we have much better methods and technology to deal with this variability.

The role of data integration in automation

Automation enables execution of processes without human intervention. In my experience, one of the most common things people look for when thinking about information systems is more automation. I believe it is crucial to maintaining a competitive balance. Automation lowers cost, increases efficiency, and improves accuracy. We need all these to be able to compete within both domestic and global markets.

As soon as we add any outside environment, we must then include some manner of data integration. This means that any process that requires communications with outside environments must have effective data integrations.

Companies that do not have the ability to support electronic messaging present a particular challenge. When trading with China, for example, we not only have a challenge with communications methods, we also have additional requirements such as tracking of materials on the water. It is important to be able to integrate these companies into the process. What you can expect are more web-based portals that will directly integrate into the messaging systems. This means the ability to communicate information in a usable manner will be available to the general Dynamics AX population. We are talking about true all-encompassing B2B systems.

There will also be more products that sit above ERP and other delivery systems. Such solutions as distributive order management systems (DOM) are already available today. These systems allow for the interrogation and the decision matrix to provide the best overall result for such things as order fulfillment. Again, it is based on integration to many different systems and providing a centralized data structure that supports the information needed to route the information to the correct party.

Historically the problem with automation has been the costs and the risk. The cost has been historically beyond the reach of many midsize companies. The challenge to the industry is to lower the cost and simplify the development and maintenance of data integration. I believe the landscape is changing and this level of integration is currently available using such things as better integration methodologies (Integration Frameworks). Dynamics 365 for Opertions will also help drive some level of standardization. This is because the changing of the tool sets, the higher cost of customizations, and hopefully the user community demanding a better integration solution set.

Dynamics 365 materially changes integration

With Dynamics 365 for Operatinons we have had to change how we looked at data integration. The traditional methods we used are no longer valid. We will not have the same accessibility to our data. Microsoft has provided improved web services and is working on other methodologies like Logic Apps. The bottom line is that the situation has changed and we all must change with it. In the near term, there will be a higher cost of data integration based on the learning curve and other limiting factors (e.g., additional overhead for such things as code promotion).

These situations could also provide a real opportunity to create more standardization. Imagine if you will, better and simpler integration technology widely available. It will not eliminate the need for supporting some unique requirements, but could provide a level of standardization we have yet to see. EnVista has created its own integration framework for all versions of Dynamics AX from 2009 forward, including Dynamics 365. We believe that standardization across all versions is important to the continuing success of the product line. What drove it was our desire to standardize and to be able to seamlessly migrate integrations from version to version. There also was a desire to reduce the cost of integration through a modular approach to the process.

Conclusion

I believe that automation was never so accessible as it is today. We as an industry must embrace more standardization and how we communicate with the outside world. The biggest cost driver is truly the data integration.

We also must not look backwards. Data integration is more than EDI. It can encompass any information that is consistent and has all the elements needed to initiate an outcome. This includes virtually any known messaging standards including spreadsheets, emails, and even scanning through optical character recognition (OCR).

It really is an exciting time for data integration. It will be very interesting to see where we are at in the next 10 years. It took us over 30 years to get this far.