Azure Updates: Distributed Hybrid Infrastructure; LLMs; FinOps; Latency metrics

In its latest Magic Quadrant for Distributed Hybrid Infrastructure, Gartner recognized Microsoft as a Leader. Microsoft offers a variety of edge-to-cloud tools such as Azure Arc, AKS, and Stack HCI. Microsoft places closely in the analyst firm's assessment alongside with VMware, Oracle, and AWS, according to the analyst firm's assessment. To simplify global operations, customers like SKF, Carnival, and COOP are using Azure technologies like Stack HCL with machine learning, SQL managed instance, PostgreSQL, and AKS.

Principal software engineer Kushal Datta and several co-authors discussed the ways that Microsoft is helping to power large language models. Recently, a GPT-3 LLM model and its 175 billion parameters were trained to completion in four minutes on 1,344 ND H100 v5 VMs. “[C]reated for performance, scalability, and adaptability, the Azure ND H100 v5-series offers exceptional throughput and minimal latency for both training and inferencing tasks in the cloud and offers the highest quality infrastructure for AI,” Dutta explained. Also on the topic of AI, CVP Gregory Buehrer discussed the enterprise large language model lifecycle. Azure AI offers an expansive model catalog, allowing developers to choose and deploy foundation models from providers like Hugging Face, OpenAI, and Meta. Buehrer explained several different development “loops” such as ideating, building, operationalizing, and management.

Modern services management solution architect Joseph Marino announced that Microsoft is now a FinOps Certified Service Provider. The FinOps Framework was created to optimize cloud costs. “At Microsoft, our commitment to you extends throughout the entire service lifecycle. Whether you are Unified Enterprise Support customer, receiving Proactive Services, or a Microsoft Industry Solutions Delivery (ISD) customer, receiving modernization and enabling innovation for the Microsoft cloud, we are here to provide the expertise and guidance you need to meet your FinOps goals,” Marino wrote.