Microsoft’s “era of copilots” will grow to cover development tools and services like GitHub, Power Apps, and Power Automate, productivity tools like Viva and Microsoft 365, Dynamics 365 business applications, and more. At Build 2023, the company is announcing more areas where Copilot will play a role: Power Pages, Power BI, Microsoft Fabric, and Windows.

And the company is announcing new features that will help developers build their own copilots and AI applications. Plugins will allow Microsoft’s copilots (the company is using the small ‘c’ here) to interact with other software and services.  The company now uses the term “copilot” not just as a brand name but to refer to “an application that uses modern AI and large language models to assist you with a complex cognitive task.”

Microsoft is highlighting development tools and guidance at Build for third parties that appear to share many of the same capabilities Microsoft’s R&D teams have used for upcoming tools like the Dynamics 365 Copilot modules as described by Microsoft VP Walter Sun recently.

In a statement for Build, Microsoft wrote:

Microsoft is adopting the same open plugin standard that OpenAI introduced for ChatGPT, guaranteeing interoperability across ChatGPT and the breadth of Microsoft’s copilot offerings. That means developers can now use one platform to build plugins that work across both business and consumer surfaces, including ChatGPT, Bing, Dynamics 365 Copilot, Microsoft 365 Copilot and Windows Copilot.

Microsoft’s AI development framework will help developers “build their own copilot” by supporting plugins that “augment the capabilities of AI systems by allowing them to retrieve real-time information, incorporate company or other business data, perform new types of computations and safely take action on the user’s behalf.”

In short, they explained, a plugin bridges the gap between a large language model trained on public data and a company’s private data and content. Microsoft offered the example of a travel booking copilot that uses both a public LLM and a backend travel booking system make travel plans that meet corporate rules. Another example: connecting a law firm’s Microsoft 365 Copilot with internal legal documents that are already specifically encoded, indexed, and stored to allow for search. In that case, the developer can ensure that the copilot plugin puts the proper rules and controls in place, according to the company.

Microsoft explained that developers will be able to use plugins across Microsoft’s copilots:

Microsoft is releasing a set of capabilities to facilitate the creation of plugins that work across its copilot surfaces. Visual Studio Code, GitHub Copilot and GitHub Codespaces will make it simple for developers to create, debug and deploy new plugins, for example, and Azure AI will add capabilities to run and test plugins on private enterprise data. Once created, these plugins will work across Microsoft's Copilot experiences.

Microsoft CTO Kevin Scott described the broad development process for a copilot in a series of steps. It requires a generative AI model like GPT-4 on Azure OpenAI Service. It needs a meta-prompt, or description of its role and operation. It needs a way to track conversations with users. And it needs access to the right data and services. More complex tools will also require orchestration, Scott explained, like in the case of the more complex steps required to write a whole book.

The introduction of a plugins model comes as Microsoft makes a range of Copilot product announcements at Build 2023, including an expanded this that includes Copilot in Power BI and Copilot in Power Pages in preview, Copilot in Microsoft Fabric, available in preview soon, and Windows Copilot, which will start to become available for preview in June.