Advancing Agentic AI Security to Ensure Responsible AI

Introduction

Agentic AI is the next evolution of AI, holding transformative potential for users and businesses. Compared to generative AI, AI agents can take actions, adapt to new information, and interact with other agents and systems to complete tasks on behalf of humans. While use cases are rapidly evolving, agents are already transforming the workplace through coding assistants, customer service agents, and automating enterprise productivity workflows.

With Agentic AI systems’ ability to autonomously execute multi-step tasks, make decisions, and interact with infrastructure and data, this drives a major paradigm shift, and organizations must adapt rapidly.

Unlike traditional functioning models that operate in stateless request-response patterns, agentic AI systems introduce autonomous capabilities, persistent memory, tool orchestration, identity and agency challenges, and external system integration, expanding the risks that organizations must address.

A major risk is an agents’ access to sensitive data and ability to make changes to their environment, such as updating a customer database or making a payment, are double-edged swords.

In addition in the Age of Identic AI, where personal agents act increasingly on a human’s behalf as coined by futurist Don Tapscott, we may be unleashing an insidious risk that may leave us increasingly vulnerable to fraud, surveillance and succumbing to living an agent’s swarm dream world.

As we increasingly move towards deploying multiple agents with complex interactions, outcomes also become more unpredictable. Humans must remain accountable and properly manage these risks. In the last three months alone, there have been numerous announcements, including:

Viral open source personal AI Agent OpenClaw (formerly Clawdbot / Moltbot). This agent has been designed as an autonomous assistant that can run tasks, manage calendars, send messages, and automate workflows.
Moltbook – Launched January 28, 2026 as a “AI agent-only social network” where thousands of agents interact autonomously.
Edge AI agent frameworks and SDKs – NXP released the eIQ Agentic AI Framework on Jan 6, 2026 — not a personal assistant but a platform enabling autonomous agents on edge devices.
Multiple community and product updates — e.g., OpenAI Health & Jobs Agents, extended agent updates seen in community tracking — though not official commercial releases.

Other major players to be aware of pursuing agentic AI developments are:

OpenAI – Developer of GPT models and agents; community tools like ChatGPT agents and private beta agents like Aardvark.
Anthropic – Behind Claude models and integrated agent workflows (e.g., agents in ServiceNow platforms).
Google – Through Gemini models and associated agent frameworks / IDE experiences (e.g., tools like Antigravity).
Microsoft / Azure – Building agentic security frameworks and enterprise integrations (e.g., ambient agent security tooling).

Deeper Dive into Agentic AI Agents

Agentic artificial intelligence (AI) systems are self-managing AI systems that can plan, execute, critique, and iterate across multiple steps to achieve specified objectives. These systems represent a significant evolution from traditional AI systems, moving beyond simple pattern recognition and predetermined responses to demonstrate increasingly sophisticated abilities to understand context, formulate plans, and take independent actions to achieve specified objectives. Development of these systems brings new capabilities and opportunities for organisations and users. Agentic AI systems are systems that can plan across multiple steps to achieve specified objectives, using AI agents. At this stage of agentic AI, there is no overall consensus on what defines an agent. However, there is agreement that they contain common features:

agents usually possess some degree of independent planning and action-taking (e.g., searching the web or creating files) over multiple steps to achieve a user-defined goal
Most agents are built on language models, and may use diverse model types like small models (SLM), large models (LLM), or multimodal large language models (MLL) to make decisions and complete tasks.

What are the core components of an AI agent?

Every AI Language model application has key components that are integrated to enable the agent to perform its specific functions:

Base Model: The brain of the agent, whether it’s a Small Language Model, Large Language Model, or Multi-Modal — serves as the central reasoning and planning engine of the agent. It processes instructions, interprets user inputs, and generates contextually appropriate responses.
Instructions: Natural language commands that define an agent’s role, capabilities, and behavioural constraints, e.g., a system prompt for an LLM.
Memory: Information that is stored and accessible to the LLM, either in short or long-term storage. Sometimes it is added to allow the model to obtain information from previous user interactions or external knowledge sources. An agent uses the model, instructions, and memory in similar ways to an LLM-based app.
Planning and reasoning: The model is trained to reason and plan, meaning that it can output a series of steps needed for a task.
Tools: Tools enable the agent to take actions and interact with other systems, such as writing to files and databases, controlling devices, or performing transactions. The model calls tools to complete a task.
Protocols: This is a standardised way for agents to communicate with tools and other agents. For example, the Model Context Protocol (MCP) has been developed for agents to communicate with tools, whereas the Agent2Agent Protocol (A2A) defines a standard for agents to communicate with each other
Multi-agent setups. In an agentic system, it is common for multiple agents to be set up to work together. This can sometimes improve performance by allowing each agent to specialise in a certain function or task and work in parallel.

Three common design patterns for multi-agent systems are:

Sequential: Agents work one after another in a linear workflow. Each agent’s output becomes the next agent’s input.
Supervisor: One supervising agent coordinates specialised agents under it.
Swarm: Agents work at the same time, handing off to another agent when needed

How agent design affects the limits and capabilities of each agent. While each agent may have the same core components, the design of each component can significantly affect what the agent can do. It is generally helpful to distinguish between two concepts when considering what an agent can do – often referred to as the Agent Action-space (or authority, capabilities).

The range of actions the agent is permitted to take is determined by the tools it is allowed to use, and the transactions it can execute. There are different Agent context protocols to consider in developing a multi-agent system. Some references are:

What are New Agentic AI Security Frameworks to Manage Risk

Over recent months, new Agentic AI security frameworks have been rapidly developed. Below is a summary of some robust frameworks to guide customers and that are also referenced by leading standard bodies, system integrators, analysts, auditors, etc

Amazon – The Generative AI Security Scoping Matrix – This framework identifies five security disciplines that span the different types of Gen AI solutions and provides helpful guidance to classify different risk categories of agents. See here for details. The framework covers:

Governance and compliance. The policies, procedures, and reporting needed to empower the business while minimizing risk.
Legal and privacy. The specific regulatory, legal, and privacy requirements for using or creating generative AI solutions.
Risk management. Identification of potential threats to generative AI solutions and recommended mitigations.
Controls. The implementation of security controls that are used to mitigate risk.
Resilience. How to architect generative AI solutions to maintain availability and meet business SLAs.

OWASP -Security Project for Large Language Model Applications . This is the repository for the OWASP Top 10 for Large Language Model Applications. However, this project has now grown into a comprehensive OWASP GenAI Security Project – a global initiative. See here for details

Singapore – Emerging Tech and Research for Agentic AI. The Singapore framework and set of guidelines was released in January 2026 and it is a living framework. The research recommendations are:

Assess and bound the risks upfront. Organisations should adapt their internal structures and processes to account for new risks from agents. Key to this is first understanding the risks posed by the agent’s actions, which depend on factors such as the scope of actions the agent can take, the reversibility of those actions, and the agent’s level of autonomy. To manage these risks early, organisations could limit the scope of impact of their agents by designing appropriate boundaries at the planning stage, such as limiting the agent’s access to tools and external systems. They could also ensure that the agent’s actions are traceable and controllable through establishing robust identity management and access controls for agents.
Make humans meaningfully accountable. Once the “green light” is given for agentic AI deployment, an organisation should take steps to ensure human accountability. However, the autonomy of agents may complicate traditional responsibility assignments which are tied to static workflows. Multiple actors may also be involved in different parts of the agent lifecycle, diffusing accountability. It is therefore important to clearly define the responsibilities of different stakeholders, both within the organisation and with external vendors, while emphasising adaptive governance, so that the organisation is set up to quickly understand new developments and update its approach as the technology evolves. Specifically, “human-in-the-loop” has to be adapted to address automation bias, which has become a bigger concern with increasingly capable agents. This includes defining significant checkpoints in the agentic workflow that require human approval, such as high-stakes or irreversible actions, and regularly auditing human oversight to check that it remains effective over time.
Implement technical controls and processes. Organisations should ensure the safe and reliable operationalisation of AI agents by implementing technical measures across the agent lifecycle. During development, organisations should incorporate technical controls for new agentic components such as planning, tools and still-maturing protocols, to address increased risks from these new attack surfaces. Before deployment, organisations should test agents for baseline safety and reliability, including new dimensions such as overall execution accuracy, policy adherence, and tool use. New testing approaches will be needed to evaluate agents. During and after deployment, as agents interact dynamically with their environment and not all risks can be anticipated upfront, it is recommended to gradually roll out agents alongside continuous monitoring after deployment.
Enable end-user responsibility. Trustworthy deployment of agents does not rely solely on developers, but also on end-users using them responsibly. To enable responsible use, as a baseline, users should be informed of the agent’s range of actions, access to data, and the user’s own responsibilities. Organisations should consider layering on training to equip employees with the knowledge required to manage human-agent interactions and exercise effective oversight, while maintaining their tradecraft and foundational skills.

Conclusion:

While existing governance principles for trusted AI, such as transparency, accountability, and fairness, continue to apply, they need to be translated into practice for agents. Meaningful human control and oversight need to be integrated into the agentic AI lifecycle. Nevertheless, a balance needs to be struck as continuous human oversight over all agent workflows becomes impractical at scale.

These emerging frameworks discussed in this article need to be further validated, but they are excellent sources to help organizations evolve their governance practices beyond basic GenAI to understand Agentic AI can be applied within a risk construct. With careful consideration, we can introduce Agentic AI practices successfully, without ensuring humans are in the loop – the consequences may be more than we envisioned.

A Personal Perspective

I have spent the last five years researching and and won recently a top CEO in North America Award for advancing Responsible AI practices not only in our AI solutioning, but also in my consistent mission of educating board directors and executives worldwide with leading colleges, universities or institutions like the United States Artificial Intelligence Institute (USAII).

In May 2026, Cathy Cobey EY’s global AI Co-Lead and I are releasing our new book, The AI Precipice: An Executive Guide to Balancing Innovation and Safety. Our book goes into robust detail of the AI strategy, governance, control requirements to ensure human in the loop (HIL) practices are integrated into AI practices. Our comprehensive research provides executives with rich scenarios and risks if they don’t build strong responsible AI decision making muscle and align across diverse stakeholder ecosystems to build a world designed for humans vs being controlled by agent swarms that take over, a validated risk by top AI security and safety researchers – yet we keep blazing the trails ahead and still remain without international security and safety aligned legal and regulatory systems.

Having spent the last thirteen years in designing and building complex AI models, predictive, multi-model and more recently generative AI LLM methods, I have experienced first hand the risks and the realities of model drifts, model hallucinations, and the naivety of senior executives putting far too much blind faith in their AI solutioning without rethinking the underlying operating processes and the jobs to be done.

One approach we took in building our SalesInsights AI product, a predictive forecasting tool that analyzes Salesforce datasets and can achieve 95% + predictive accuracy, (See case studies) is that by tracking every sales deal change and showing the completeness of the data in the AI model can revolutionize how sales forecasting is done and literally free up sales leaders to reinvest 10 % of their time in what really matters – talking to real customers. We just released a second new product, called MoodInsights which is using GenAI and NLP methods and its incredible what can be crystalized with the right prompt architecture to mine volumes of data and improve the intelligence sense making. At the same time, human in the loop (HIL) is needed to ensure the classifiers are performing consistently and not misclassifying or ignoring insights that could have severe consequences. Even a 1% error rate in high risk situations is too high – so vigilance is needed on managing risk in building GenAI solutions let alone having swarms of Agents operating.

We cannot be naive about the power of Agentic AI and it may also be greater than our own business knowledge – so we must THINK, a phrase IBM created in the 1980s when their own organization was not attune to the realities of the changing technology landscape.

I often recommend to our customers to slow down as there are so many AI failures and AI investment waste that could be deployed to so many more worthy experiences. I often ask when was the last time you sat down with real customers or with employees who increasingly are fearful of AI and listened to their voices without counter perspectives – simply hold their views with respect, and without judgement, applying perhaps appreciative inquiry methods, a skill that needs more cultivation to increase collaborative decision making.

We have also learned how incredibly messy humans are in their data practices and the first order of business before doing any AI development is to shore up a comprehensive audit of an organization’s data health across its operation and take seriously that unless data is treating like a strategic asset, like its literally liquid gold – many companies will simply cease to evolve and the speed of AI evolution should be enough to get the hair on the back of everyone’s neck to stand up.

This is also a period of time where the trust imperative is crucial in building agentic AI systems.

If you are unsure of how to design, develop and sustain your AI investments, especially in Gen and Agentic AI methods, do reach out. We have a track record for building responsible AI systems that sustain and deliver over 100% ROI consistently – versus the industry average of less than 10%.

Our approach is not rocket science – it simply starts with pragmatism – of analyzing the problem, understanding the work that needs to be done and determining if the problem is justified in using AI methods or not. We also place an incredible emphasis on the stakeholders involved in the design impact and ask the hard questions that many do not seem to ask. Change management and sustaining communication that uplifts an organization will align people to shift mindsets – but ignoring the people equation is a big mistake. Culture always trumps and eats for lunch new initiatives that are not well though through and add value. If you are following the results of AI initiatives over the past 10 years – you will already appreciate how imperative it is to increase focus on what motivates people to change and ensure there is more WIFME (What’s in it for me?).

Not every business process is worth investing in AI methods and the sooner we understand that if we want to shift the tide of over 85% of AI projects fail, each AI use case invested in should always demonstrate a value – otherwise invest time in building relationships with real humans, go into a business meeting with no PowerPoints or technology – just tell stories and rediscover the value of a real human to human conversation – you might be surprised of how you feel as a human – an organism that requires social connections to thrive.

This article was written to help educate board directors and C-Leaders increase their understanding of Agentic AI. The research sources below can further guide leaders to increase their agentic AI knowledge and the risks associated with them.

I highly recommend everyone read the AI Safety Report, as currently, the methods for training models to resist malicious attacks continue to improve in 2025, with prompt-injection attack success rates declining over time. However, tests show that sophisticated attackers can still bypass safeguards around half of the time when given 10 attempts.

THINK: What happens with agent swarms and there are more access points to hack into and the impact of human safety? If implementing agents ensure you are world-class in cybersecurity controls or PAUSE.

#AI #AIAgent #AgenticAI #AIFrameworks #AIFraud #AIGovernance #AIRisks #AISecurity #AIStrategy #Amazon #Anthropic #AWS #Bain #CSA #Cybersecurity #EY #GenAI #GenerativeAI #Google #IBM #InfoSys #LLM #LargeLanguageModel #Meta #Microsoft #McKinsey #OpenAI #OWASP #ResponsibleAI #SalesChoice #SalesInsights #Salesforce #WorldEconomicForum #WEF

Research Sources:

AWS, AWS Agent Evaluation • Anthropic, Demystifying evals for AI agents

Bain, Building the Foundation for Agentic AI

Brown, Aron, and Saner, Matt. (Nov 21, 2025). Amazon: The Agentic AI Security Framework

Carnegie Mellon University, The ‘Who’, ‘What’, and ‘How’ of Responsible AI Governance

Cloud Security Alliance, Agentic AI: Understanding Its Evolution, Risks, and Security Challenges

Cobey, C. Gordon, C. The AI Precipice: An Executive Guide to Balancing AI Innovation and Safety.

CSA and FAR.AI , Securing Agentic AI: A Discussion Paper

CSA, Draft Addendum on Securing Agentic AI

EY, Building a risk framework for Agentic AI

EY, Building a risk framework for Agentic AI Agent limits and agent identity Meta, Agents Rule of Two: A Practical Approach to AI Agent Security

Gordon, C. AI Insider Newsletter (Linkedin)