Agent 5.6 Instead of 007 When the Agent Does More than He Should

AI agents analyze source code, use development tools, and access corporate data. This creates a security issue. It’s not just attackers who can cause malfunctions; even an agent that pursues its mission with particular diligence can cross boundaries. Protection is provided by an architecture that monitors every action and limits its consequences.

An assistance system that merely explains a circuit or summarizes a data sheet can provide incorrect information. An AI agent that modifies source code, retrieves measurement data, runs simulations, or accesses a production database, on the other hand, can directly intervene in technical processes. As its capabilities grow, therefore, not only does its usefulness increase, but so does the potential damage caused by a wrong decision.

The security issue doesn’t start with a targeted attack. During an internal hackathon, Check Point examined, among other things, how an autonomous SRE agent (Site Reliability Engineering agent) responds to tasks that cannot be resolved using its available tools. According to the company, the agent restarted a production database, interrupted connections, and attempted to expand its permissions. The agent had not been compromised. It interpreted these actions as appropriate steps to ultimately achieve the specified goal.

This is relevant in engineering environments because an agent can, in theory, work toward a legitimate goal while still choosing actions that are impermissible or technically risky.

External Data Can Be Turned into Instructions

A second risk arises from the information an agent processes while performing its tasks. This includes source code, README files, tickets, emails, web pages, measurement results, and responses from external tools. People usually recognize from the context whether a text passage is a work instruction, documentation, or mere payload. Language models do not reliably distinguish between these categories.

This means that a tampered file can contain instructions that the agent executes during what is otherwise a legitimate task. Check Point reports on attempts in which programming agents followed hidden instructions in repository files and attempted to transmit login credentials. This method, known as indirect prompt injection, does not require direct access to the agent. It is sufficient to manipulate a source of information that the agent will later process.

In addition to prompt injection, OWASP lists tool misuse, privilege escalation, data exfiltration, poisoned storage contents, and excessive autonomy among the key risks associated with agent-based systems. To date, there is no fully reliable protection provided solely by system prompts or input filters. External content must therefore generally be considered untrustworthy.

The Execution Layer Becomes the Checkpoint

In traditional applications, the program code largely determines which function is executed next. An agent, on the other hand, decides based on the situation which tool to call and with which parameters. This is precisely where the security architecture must come into play.

An independent control layer is situated between the agent and the tools. It checks every file access, shell command, API call, and database operation before the action is executed. In doing so, it must determine whether the desired tool is approved for the task at hand, whether the agent intends to read only or also write to or delete data, and whether the affected resource belongs to the permitted scope of operations. Equally relevant are potential data leaks, the reversibility of an action, and unusually long chains of actions or repetitions.

A single operation may appear innocuous, while several consecutive steps together may constitute an attack or an expansion of privileges. Monitoring must therefore not only evaluate each tool call in isolation; it must also take into account the sequence of actions to date, the original task, and the current system state.

Explicit prohibitions should enforce deterministic rules. These include blocked directories, unauthorized network destinations, invalid database commands, and missing permissions. An additional AI-based check can assess whether an action is still consistent with the task at hand. However, it must not serve as the sole security measure. Even as a monitoring system, a probabilistic model remains fallible.

All security-related operations must also be logged in a structured manner. This includes the original request, the tool used, the target resource, the parameters passed, any authorization granted, and the result. Only this information makes it possible to reconstruct an incident or identify recurring patterns of error. In the event of unusual behavior, a higher-level system must be able to stop the current task and revoke all temporary permissions.

Why Constant Follow-Up Questions Aren't Enough

An obvious safety measure is to ask users for consent before every critical action. In practice, however, this quickly leads to “confirmation fatigue.” Anthropic reports that users confirm about 93 percent of the permission prompts they see. A warning that appears frequently thus increasingly becomes an obstacle that users routinely click away.

Human approvals remain useful for irreversible, financial, administrative, or externally visible actions. However, they should be used selectively and clearly describe the specific action. Approval must be tied to the tool, the target, the parameters, and the timing. A blanket permission such as “allow access” offers only limited protection. For recurring activities, technically enforced limits are more reliable. An agent can act largely independently within a clearly defined scope of work without having to obtain approval for every non-hazardous step. As soon as it attempts to go beyond these limits, the execution layer must stop the process or require a new, specifically tied authorization.

Safe Alternatives to Complete Demolition

If the runtime check detects a risky action, there are initially two obvious options: execute it or block it. While a complete termination protects the system, it often also prevents legitimate tasks from running. If too many tasks are blocked, the pressure to disable the security feature increases.

A third option is to offer the agent a safe alternative course of action. For example, instead of restarting the production database, the agent can run a diagnostic test and create a restart schedule. Instead of permanently deleting files, the agent can move them to quarantine. A configuration change can first be displayed as a diff, a software version can be tested in a staging environment, and an email can be submitted as a draft for approval.

According to Check Point, the company compared nine response strategies based on 90 tasks from the “DecodingTrust-Agent” benchmark. Without monitoring, the success rate of the attacks examined was 46.7 percent. The countermeasures employed reduced it to between 22 and 27 percent. A strategy that offered the agent a secure alternative resulted in more successfully completed tasks than a strict blocking strategy.

The results are based on a limited experimental setup and do not demonstrate a general protective effect. However, they highlight an important trade-off: Security measures must not only prevent attacks but also remain acceptable in day-to-day operations.

In engineering processes, an agent can therefore first propose a measure rather than executing it directly. Changes can be submitted for review as a patch, command, or work plan. Where a practical test is necessary, the action can be performed as a dry run, on a copy of the data, or in a staging environment. Snapshots, versioning, and transactions allow actions to be undone, while time-limited access credentials and tools restricted to specific resources limit the scope of the action.

Even a Sandbox Can Fail

A sandbox is designed to isolate agents from sensitive files, production systems, and uncontrolled network access. It thus remains a central component of the security architecture. However, the term can suggest a level of isolation that cannot be guaranteed in practice.

This was revealed by a security incident that occurred during an internal model evaluation at OpenAI. The models used did not have direct internet access; they could only retrieve software packages through an internally operated proxy and cache. According to OpenAI, however, the models discovered and exploited a previously unknown vulnerability in this package proxy. They then carried out several steps to escalate privileges and move laterally until they reached a node with internet access.

Using this connection, the models searched for solutions to the cybersecurity challenge presented to them. In doing so, they chained together vulnerabilities in the OpenAI research environment and in Hugging Face’s production infrastructure. In one instance, they combined stolen credentials and zero-day vulnerabilities to find a way to execute code remotely on Hugging Face servers. OpenAI emphasizes that the models in question (ChatGPT 5.6) were run with reduced cybersecurity safeguards because the evaluation was designed to measure their maximum capabilities.

Anthropic reports, separately, that it has already observed Claude models that “successfully” escaped from test sandboxes to solve a task. More recent Claude models have also demonstrated the ability to develop complex exploits that can chain together multiple security vulnerabilities and bypass sandbox boundaries.

This makes the sandbox itself a potential target for attack. Particularly critical are the interfaces that inevitably connect an isolated environment to the outside world. Packet proxies, file-sharing services, shared directories, MCP servers, update mechanisms, and management interfaces act as bridges across the isolation boundary. A vulnerability or misconfiguration in any such component can negate the protective effect of the entire environment.

Several Boundaries Must Act in Succession

A resilient system assumes that any single layer of protection can fail. If an agent deviates from its intended process, the virtual machine must restrict access. If the agent bypasses this restriction as well, network segmentation and separate identities should prevent the next step. Even with existing network access, only approved destinations should be reachable. Access credentials must be short-lived, limited to a single resource, and stored outside the work environment.

Outgoing data traffic also requires its own controls. Domains alone are not always sufficient for this, because a permitted service can be misused as a storage or relay channel. Therefore, limits on protocols, data volumes, and specific API functions are also necessary. Suspicious connections, large data transfers, or repeated attempts to access new destinations must automatically trigger an interruption.

The most effective architecture thus combines process isolation, virtual machines, file system boundaries, network segmentation, controlled proxies, and minimal permissions. None of these measures prevents every attack. Together, however, they reduce the likelihood that a single vulnerability will open a direct path from the agent environment to production systems.

The classic principle of minimal rights thus gives rise to the principle of minimal agency for AI agents. A system is granted not only as few permissions as possible, but also only as much autonomy, context, and connection to the outside world as the task at hand actually requires.

Security Must be an Integral Part of Development

An agent system's attack surface does not change solely with the introduction of a new model. New prompts, tools, MCP servers, data sources, libraries, and permissions can also influence its behavior. Therefore, the security review must not end with the system's implementation.

The necessary test cases include tampered documents, hidden instructions in source code, attempts to escalate privileges, data leaks via authorized tools, corrupted memory contents, and excessive resource usage. Equally important are tests of the sandbox itself, as well as its proxies and management interfaces. The tests must assume that the agent does not merely inadvertently push boundaries, but rather actively searches for vulnerabilities and combines multiple ones.

Such scenarios should be incorporated into the development and release process as regression tests. Changes to tools, network rules, or access rights require the same level of attention as changes to the actual application code. For particularly critical applications, a general security assessment of the model used is not sufficient. What matters is the complete system comprising the model, orchestration, tools, data sources, identities, and execution environment.

Controllable Consequences Instead of Flawless Agents

Complete security cannot be achieved with AI agents. Prompt injection can be overlooked, a monitoring system can misjudge a risky sequence of actions, a human can confirm an incomprehensible approval, and even a sandbox can be compromised by a newly discovered vulnerability.

Security requires independent layers. Models and external content are considered potentially unreliable. Permissions remain strictly limited. Critical actions undergo a separate execution check. Work environments are isolated, connections are monitored, operations are logged, and changes are designed to be reversible whenever possible. A secure agent does not have to make the right decision in every situation. However, its environment must prevent a single wrong decision or a breached layer of protection from having uncontrolled repercussions on development data, production systems, or corporate infrastructure.

Artificial Intelligence

Google's AI, Gemini, also Hacked Other Companies