The AI Agent Confidence Gap: Risks, rewards, and best practices when working with AI agents

Application access control is an essential part of business readiness. But how does it work in practice? Delinea Fastpath recently published findings from a new survey of 1,000 business leaders examining how their organizations are governing access using foundational controls, and how they are approaching fresh uncertainties brought on by the rapid proliferation of AI capabilities, particularly AI agents. (Note: this article is sponsored by Delinea Fastpath.)

The data reveals surprising patterns for business readiness and risk. Despite high levels of confidence about the ability to withstand audits nearly half witness the same audit findings reemerging year after year. 

Leaders assess confidence in access controls

The expansion of business app environments results in sprawling permissions and connections between systems. 95 percent of respondents express confidence in the access controls for their business applications, but 51 percent also report recurring audit findings or ongoing findings. 

The divergence raises questions around close out issues such as control deficiencies, sensitive data exposure, and compliance violations, the report suggests. 

The finding could indicate a range of audit implications, from ledger transactions to user security or segregation of duties, and it raises questions for IT, notes Dag Calafell, a Microsoft MVP and Managing Director for Solution Development with MCA Connect, a Grant Thornton US company. (MCA Connect was not involved in the survey or report.)

“IT is confident that the system can support all the controls that are necessary and they have set up the system security appropriately, however audit findings may show differently,” Calafell told MSDW. “I believe employee turnover and the lack of documented processes can lead to audit findings which would have been avoided otherwise.”

“It is a human tendency to believe we are in better shape than is many times the case,” said Tom Doran, a partner at VCIO Global. “No one wants to acknowledge they don't understand or are not paying close attention to security and access control. The first step is to engage either an MSP or utilize a tool to perform a complete network scan to include all end points.” (VCIO Global was also not involved in the survey or report.)

According to Delinea Fastpath, excess access hampers oversight. Even as automation grows more common, it often fragments access while governance struggles to keep up with the growth. Companies need to be cautious about AI agents outputting incorrect information—particularly incorrect financial information—as a result. Business leaders can close the gap between confidence and access findings by triaging and addressing open findings, particularly those that have been open for a long time.

Traceability for AI agents

Approval, traceability, and permission limits emerged as significant themes throughout the report. AI agents are already widely deployed, the authors found, and they do tasks that go far beyond reading data. Depending on the use case, agents can do anything from executing transactions to managing approvals or accessing sensitive data and handling configurations. 

Respondents to the survey noted problems related to access to sensitive data, approving transactions, changing user permissions, altering app configurations, and changing business data. A remarkable 84 percent of respondents said that they have at least one AI-related business apps issue and 44 percent said AI deployment is outstripping controls.

“The lack of traceability of agents is one of the top reasons AI is not being adopted,” Calafell said. “Microsoft is addressing this need with Agent 365 and log analytics; however, those tools require additional knowledge in order to set up correctly, and intentionality in reviewing the outputs.”

Based on his experience, Doran sees a similar pattern. “Many of our clients are in a race to deploy some form of AI. We push back on speed and highlight the many issues that can arise if we do not first take into account three items: 1. Data governance, 2. Security and access control, 3. What is the goal of the AI tool being requested. AI for the sake of AI is not much more than parlor tricks and introduces many significant security concerns. A purposeful controlled approach is best practice.”

Managing application access governance and agent ownership

Organizations face pressing questions about what tasks AI agents should be allowed to do autonomously. Delinea Fastpath found that although controls are becoming more automated, problems with ownership and accountability for application access governance are commonplace. When governance is siloed, this makes it difficult to discover and monitor the permissions and actions of human users and AI agents.

Thirty eight percent of respondents said that IT controls access governance by default, resulting in limited visibility for business app owners about who has access. Often ownership records are siloed across teams or sprawl across multiple spreadsheets and trackers. Governance demands are growing along with application scope. 68 percent of respondents noted a growth in the number of users with high-level access while 58 percent reported delays in remediating access control related audit findings. 

Calafell has observed that the most common autonomous agent scenarios are surfacing in customer service: responding to customer inquiries, entering in orders, and providing order status. Other sensitive tasks within an organization still carry risks too high for most teams, he said. 

“There are plenty of sensitive areas which require human approval, such as journal postings, credit limit adjustments, and large capital expenditures – areas where the cost of inaccuracy is too high, or the way of rolling back what an agent has done is too painful. All agents acting autonomously must be explainable, observable, audited, and understood before they can be trusted,” he said.

For organizations puzzled by agent ownership decisions, simply asking more questions, and the right questions, may help to resolve the issue, the Delinea Fastpath report’s authors write. They also advocate for continuously identifying SoD conflicts and reining in excessive access. For example, a company could evaluate all business-critical apps, including those outside formal governance, in its access risk analysis. 

Across its findings, Delinea Fastpath found significant concerns about access controls, driven by the rapid growth in business applications, human users with access to sensitive data, and AI agents. Business leaders can rise to the occasion to manage traceability and governance for AI agents to ensure organizational control at every step in the process. 

“Once data governance and access control have been satisfactorily addressed, we are very excited about seeing agents perform work. The degree to which this work is human in the loop or fully autonomous depends on the degree of certainty related the type of actions being performed,” Doran said. “If we consider Dynamics 365 Business Central as an example, once the standard operating procedures and process documentation are in place, agents can perform tasks such as sales order processing for orders received that align with business rules, creating planning worksheets and simulated schedules…processing customer payments and posting cash, and preparing month end journals and tasks for the accounting team to review and post,” Doran said.