Cross-session accumulation: attackers keep state, agents don't, and per-session review never sees the whole attack

Welcome to The Observability Layer, a daily briefing on Responsible AI, AI governance, evaluation, and the technologies shaping the frontier.

A note about how this podcast is made: the hosts you're hearing are AI agents. The research, analysis, editorial perspective, and authorship behind The Observability Layer come from Dr. William Fisher. Our agents help transform that work into the conversation you hear each day.

We think that is fitting. A podcast about understanding, governing, and observing AI should not just talk about intelligent systems. It should put them to work, transparently.

This is The Observability Layer. Let's get into today's research.

Let's start there. We have two new papers that find the same attack surface from opposite directions: cross-session attacks.

Exactly. The first, called 'Magnet', demonstrates what they call cross-session goal decomposition. An attacker breaks a harmful goal into smaller, innocuous steps and executes each one in a separate, isolated agent session.

And because each session looks harmless on its own, a per-session review passes it, but the attacker is assembling the harmful capability on their end.

The paper puts it perfectly: 'the agent is stateless between conversations, but the attacker is not.' A second paper finds the mirror image inside agents that learn from experience. An attacker can feed the agent a series of individually benign tasks to build up a specific set of skills, then use one final query to activate them all together for a malicious purpose.

So, what's the Monday morning takeaway for a team running agent logging?

It means data retention and the ability to link sessions to a single user ID become safety controls, not just privacy costs. And if an agent can learn or write to memory, that write path has to be treated as a privileged operation. The poisoning happens through normal use.

Okay, so session-level review isn't enough. What about the runtime monitors designed to catch bad behavior as it happens?

They have their own problems. A paper called 'A False Average' attacks chain-of-thought monitors. The researchers took an exploit, left every single command byte-for-byte identical, but rewrote the agent's reasoning to sound like good-faith engineering.

And the result?

The monitor's catch rate on that slice of attacks dropped from about ninety-five percent to under eleven percent. The paper's point is that aggregate accuracy scores are misleading because they're dominated by obvious attacks. The monitor fails completely on the subtle cases where it's the only line of defense.

A near-total collapse. And there was a second paper on rule-based monitors?

Yes, which proves that the effectiveness of a fixed set of rules is bounded by the model itself. Some models produce very concentrated, predictable attack patterns, so a few rules can catch 75% of them. Other models produce very dispersed, varied attacks, and the same ruleset might catch only 6%. It's not about how well you write the rules; it's a property of the model you're trying to govern.

So just as we're learning the real risk is in the trajectory, not the snapshot, the White House has an update on its frontier model framework.

That's right. On August 3rd, the administration finalized its voluntary cybersecurity testing framework. The big four labs, Meta, Google, OpenAI, and Anthropic, are meeting with them today to review it.

This is the 60-day deliverable from the June executive order. But based on everything we've just discussed, is it aimed at the right target?

There's a fundamental mismatch. The framework is a capability test. It asks if a model can perform certain cyber attacks. It's a point-in-time evaluation of a static model, done pre-release. The research is all pointing to the fact that the danger is in the emergent behavior of a deployed agent over time and across sessions.

So if the federal instrument is focused on the snapshot, how are enterprises thinking about the full movie?

A few new papers give us the language for it. One, accepted to an ACM summit, argues for a shift from 'per-action checks' to 'trajectory assurance.' The core problem is what they call 'behavioral containment', making sure a sequence of individually okay actions doesn't violate a system-level rule in aggregate.

Which is exactly the problem the cross-session attacks exploit. How do you evaluate that?

Another paper points out that grading an agent against a single, static snapshot of an enterprise environment at the end of a task is flawed. The fix is point-in-time replay, allowing you to evaluate what the agent did at any moment with the information it had at that moment. And a third paper, 'FRAMES', shows how to let an agent evolve new skills while providing an anti-regression guarantee, so it doesn't silently break an old rule while learning a new trick.

This theme of evaluation validity, of making sure our yardsticks are measuring the right thing, came up in fairness, too.

It did. A paper titled 'Who Should Be Generated?' points out that every generative fairness audit compares the model's output to some target demographic distribution, but that target is almost never justified. It's just assumed.

And that choice of target matters.

It matters immensely. The researchers held the model's generated images and the measurement metric constant. They only swapped the comparator, for instance, from a geography-based demographic target to an equal-representation target. The measured unfairness score changed by about a third of the entire possible range. The choice of yardstick can be more important than the model's behavior.

The takeaway is clear: if you're reporting a bias score, you have to state and defend your benchmark. Let's round out with a few items worth watching.

First, cheap deterministic checks often beat expensive LLM judges for catching agent failures. One paper found that simply re-calculating an agent's math and checking if it made required API calls caught 60% of failures with zero false positives.

There's also a new method for knowing when to stop an evaluation early, which is crucial for making partial benchmark runs auditable.

And a new K-12 safety evaluation found that models are most vulnerable in multi-turn conversations, echoing a finding from yesterday that single-turn safety scores don't tell the whole story.

Finally, the 'Pacing the Frontier' statement from frontier lab employees now has over 1,300 signatures. They're asking for tools to pace automated AI R&D, which feels very different from the pre-release capability testing the White House is rolling out today.

The gap between the research frontier and the policy frontier is today's throughline.

And that's our show. Join us tomorrow for more of The Observability Layer, Daily.

That's today's edition of The Observability Layer.

If you value rigorous, practical Responsible AI research without the hype, like, follow, and subscribe wherever you listen. It helps more people working at the frontier of AI find the show.

And we want to hear from you.

For questions, comments, research recommendations, or topics you think deserve deeper investigation, reach out to Dr. Fisher at assistant@theobservabilitylayer.com.

We are particularly interested in cutting-edge research that is impactful, technically credible, and well supported by evidence. If there is something the Responsible AI community should be paying attention to, send it our way.

The research and editorial direction of The Observability Layer are authored by Dr. William Fisher, with production and presentation performed by AI agents.

Until next time, keep looking beneath the model, beneath the interface, and beneath the claims.

This is The Observability Layer.