Azure Updates: AI agent costs; Foundry; PostgreSQL; Patch window

Microsoft VP Steve Sweetman explored ways to optimize the cost of AI agents. He explained that for most AI workloads, users pick the strongest model and prompt it with everything needed for the model. AI workloads aren’t uniform because of different intent classifications, formatting, summarization, and extraction. A single outcome may be comprised of many different requests. 

Sweetman encouraged customers to use Microsoft Foundry for “four levers” at runtime. Foundry offers capabilities such as model routing, provisioned throughput, and deployment types along with caching, prompt and agent optimization, and observability. Model router capabilities direct income requests to the most suitable underlying model with model subsets that align with Azure Policy to constrain routing to an approved allow list. Currently, agents re-run the same instructions, schemas, and policy text on each turn. Prompt caching can get around this somewhat by improving latency. 

CVP and General Manager for Azure Networking Igor Sakhnov looked into what he describes as a “growing gap” between how quickly organizations remediate vulnerabilities and how fast adversaries are able to target them. The patch window has shrunk with attackers able to exploit a vulnerability the same day it is found. Organizations still need to go through multiple steps to fix a problem like identifying affected systems, validating fixes, and monitoring deployment and regressions. 

According to Sakhnov, AI is increasingly able to assist with vulnerability disclosures. Many organizations have increased visibility with threat analytics tool. While that is an improvement, it does not resolve extra validation steps needed to prevent disruption to the business. He noted that compared with more established endpoint-based controls, new network-level protections can operate around workloads to restrict access to at risk systems or segment assets. He wrote: