An initial setup note on ControlArena, trusted monitoring, and the path toward multi-agent control.
The main goal of this project is to explore how AI-control techniques can move beyond a single agent trajectory and operate across many concurrent agents. Instead of treating each model run as an isolated evaluation, I want to investigate a control plane that can observe trajectories, aggregate risk signals, coordinate interventions, and make deployment-level decisions.
Initial setup
I started with ControlArena, a framework built on Inspect AI for evaluating control protocols.
The first experiment used the APPS programming setting and a defer-to-trusted protocol:
- An untrusted policy attempts the programming task.
- A trusted monitor assigns suspicion scores to its actions.
- If suspicion exceeds a threshold, the action is discarded and regenerated by a trusted policy.
- The experiment runs in both honest and attack modes.
- ControlArena records the complete trajectories and computes safety and usefulness metrics.
Attack-mode agents are additionally instructed to introduce a hidden backdoor while preserving normal task behaviour. Docker keeps the generated code and its tests isolated from the host system.
First results
The initial five-sample-per-mode run produced:
| Metric | Result |
|---|---|
| Honest trajectories | 5 |
| Attack trajectories | 5 |
| Usefulness | 0.60 |
| Reported safety | 1.00 |
A usefulness score of 0.60 means that three of the five honest tasks were completed successfully.
The safety score needs more careful interpretation. With such a small sample, 1.00 may simply mean that the attack policy failed to produce any successful covert attacks. It does not yet demonstrate that the monitor reliably detects strong attacks. This run was primarily an end-to-end systems check: model calls, Docker execution, monitoring, trajectory logging, scoring, and analysis are now working.
Where this is going
The next step is to extend the experiment from independent trajectories to a deployment containing multiple parallel agents. That introduces a different class of control questions:
- How should risk be aggregated across trajectories?
- Can suspicious behaviour be detected only when evidence from several agents is combined?
- Should one agent’s behaviour change the permissions or monitoring applied to others?
- When should the system audit, defer, throttle, isolate, or shut down?
- How do we maintain a shared risk and audit budget over time?
The intended architecture includes parallel trajectory workers, a shared event stream, online monitors, a deployment-level risk model, and a controller capable of applying interventions across the agent population.
This is only the baseline. Larger experiments, stronger attack policies, multi-trajectory monitoring, and the first parallel-agent control-plane prototype are coming next.