Ignite 2023: Microsoft to add Copilot to mixed reality experience of Dynamics 365 Guides

View all of our Ignite 2023 coverage

Copilot is coming to the mixed reality experience of Dynamics 365 Guides for frontline workers, Microsoft announced today.

The addition of Copilot capabilities to the mixed-reality experience is intended to help workers get the needed information more quickly. The company explained the vision for the solution in a blog post:

Instead of workers needing to search through volumes of digital documents or paper manuals, Copilot enables workers to ask for relevant information using natural language and human gestures such as looking and pointing. Copilot uses generative AI to search for information from technical documentation, service records, training content and any other data sources that customers curate. Copilot then summarizes the information to provide interactive guidance through content and holograms overlaid on the equipment in operation or maintenance.

Copilot for D365 Guides is entering into a private preview today. It will be available first on HoloLens 2, with plans to later bring the capabilities to mobile devices. Capabilities of the solution will include:

• Point and ask guidance – a worker can point to an asset and ask a question about its attributes or functions. The system will also have knowledge of the parts in 3D model and can provide guidance on an assembly and its parts.
• Spatial content generation – Copilot will reduce upfront investments related to creating spatial content by using customer-defined data sources.
• Step-by-step assistance – The language capabilities of the AI model allow workers to have “progressive conversations” to walk through a progression of work steps, providing information along the way via speech, text, and 3D holograms.
• On-demand information – The copilot can use data from sensors, operational systems, and other data sources to answer questions about the environment, the objects in the environment, and related information.
• Expert knowledge integration – The copilot can be used to pull up historical information like past notes or call transcripts related to the same environment.

...

Image from Microsoft