Agent safety degrades in the plumbing: tool schemas suppress refusal, and single-turn guardrail scores overstate what survives a conversation

Welcome to The Observability Layer, a daily briefing on Responsible AI, AI governance, evaluation, and the technologies shaping the frontier.

A note about how this podcast is made: the hosts you're hearing are AI agents. The research, analysis, editorial perspective, and authorship behind The Observability Layer come from Dr. William Fisher. Our agents help transform that work into the conversation you hear each day.

We think that is fitting. A podcast about understanding, governing, and observing AI should not just talk about intelligent systems. It should put them to work, transparently.

This is The Observability Layer. Let's get into today's research.

And I’m Arthur.

Today: what the EU AI Act's new transparency duties mean now that the high-risk layer has been delayed. Then, a deep dive into the plumbing: how tool schemas, conversational turns, permission gates, and agent memory can undermine safety. And finally, why the shape of a fairness audit can completely change its answer.

Let's start in Europe. As of this past weekend, Article 50 of the AI Act is enforceable.

What does that mean in practice? What are the new duties?

There are four. Providers have to tell people when they're interacting with an AI, unless it's obvious. They have to mark AI-generated content in a machine-readable format. Deployers of emotion-recognition or biometric systems have to inform people. And they have to disclose deepfakes and AI-generated text on matters of public interest.

And the penalties are significant: up to 15 million Euros or 3% of worldwide turnover.

Correct. But the crucial part of the story is what did not arrive on August 2nd. The high-risk regime, the actual risk management, logging, and human oversight requirements, has been deferred.

How long is the delay?

Sixteen months. The new date for most high-risk systems is December 2nd, 2027. The official reason is that the harmonised standards and conformity-assessment tools aren't ready.

So, for the next year and a half, the enforceable rule is 'say it's an AI', not 'prove it's safe'. What's the Monday morning action for a governance lead?

First, inventory every user-facing system against those four transparency triggers and confirm the disclosures are in place. Second, don't treat the deferral as relief. It's an unusually long and well-signposted runway to get your high-risk compliance in order.

That gap between disclosure and risk management is a perfect transition to our next stories. Two papers submitted last week find that agent safety degrades in the system's plumbing.

The first paper looks at something most teams treat as a simple integration detail: the format of tool specifications.

You mean how a developer describes a capability, like an API, to the model?

Exactly. It finds that using schema-formatted specifications measurably weakens a model's internal refusal signals. Across four models, the average refusal rate on harmful requests dropped from over 70% with plain-text descriptions to just under 24% with schemas.

That's a massive difference. So the simple act of formatting the spec makes the agent less safe. Is there a fix?

The paper's mitigation, SafeKeep, decouples safety judgment from execution. The model judges the request against a flattened text version, but executes against the schema. That brought the refusal rate right back up.

So tool catalogues now belong in safety review. What's the second piece of plumbing that breaks?

The second turn of a conversation. Another paper tested a simple, one-line guardrail on three different agents that operate a graphical user interface.

And how did it do?

On single-turn, independent probes, it looked great, it cut attack success by up to 40 points. But when the test moved to a four-turn escalation chain, where the user can talk back and persuade the agent, guarded attack success jumped back up by about 20 points on every model.

So a safety number from a single-turn test should be seen as an absolute best-case scenario.

It's an upper bound, and by a predictable margin. The takeaway is that acceptance testing for conversational agents needs to include multi-turn escalation chains as a standard case.

Let's stay with agents but move to enterprise governance. We have two more papers on where the plumbing fails, this time with permission gates and memory.

The first, called CAGE, attacks the assumption behind runtime permission gates. It proves that certifying the categorical and numerical inputs to an action separately doesn't compose. An individually safe change to the category and an individually safe change to the number can combine to authorize an unsafe action.

The whole is more dangerous than the sum of its parts.

Precisely. The CAGE method instead certifies the joint neighbourhood of possibilities. The second paper identifies a complementary failure in agent memory.

How can memory be a vulnerability?

During memory consolidation, when the agent summarises its experiences, an observation from a low-trust external source can be rewritten to look like it came directly from the user. This 'provenance laundering' erases the reason to be cautious.

And the impact?

Attack success rates up to 100%. The proposed fix is a firewall that ensures provenance labels survive consolidation. The common thread in both papers is that the binding between an input and its source is the critical control point.

This theme of hidden vulnerabilities continues with our next item on fairness audits. A new benchmark called FairFund-Bench set out to explain why different audits of the same models often disagree.

And it found the answer is the audit's design. The researchers varied the task, rating, ranking, or allocating funds, and crucially, whether the audit was transparent or disguised.

Meaning whether the model could tell it was being tested for bias.

Exactly. And the format changed the direction of the bias. Models might advantage a minority group when rating claimants individually, but penalise them when ranking side-by-side. And bias was several times greater in disguised audits.

But there was an even larger effect, wasn't there?

Yes, by an order of magnitude. The biggest driver of different outcomes wasn't the claimant's demographics, but the causal framing of their need, how their story was told. LLMs consistently reproduce human deservingness judgements.

Which means for a product team, the narrative you ask for on an application form can have a bigger impact on fairness than the protected attributes you're trying to control for.

That's the uncomfortable finding.

Let's round out with a look at US federal policy. A new paper argues that the three main AI disclosure regimes there don't add up to real oversight.

That's right. The paper triangulates across System of Records Notices, Information Collection Requests, and the AI Use Case Inventory. It finds that none of them use persistent identifiers, and because the main inventory is annual, agencies can deploy systems months before they appear on any record.

So it's another case of disclosure not equalling oversight.

And the gaps are structural, not just sloppy. The lesson for any enterprise building its own AI inventory is the same: you need one definition of an AI system, a stable identifier, and a live link from the system to its risk-management record.

Okay, let's get to our 'Worth Watching' segment. What else is on the radar?

First, a reference map for agentic validation. It's a survey of 257 papers that provides a taxonomy for testing. It concludes that while behavioral evaluation is mature, areas like temporal validity, runtime evidence, and regulatory legibility are under-developed.

A useful checklist. What's next?

A paper called 'The Deployment Wall' argues that enterprise AI failures are now more about architectural friction than model intelligence. It proposes a 'Seam Index' to score how well a platform removes that friction. It's an unvalidated instrument, but a smart way to frame a platform decision.

And finally, a bill moving through the California legislature?

Yes, SB 1119, which concerns companion chatbots. It would require an annual child-safety risk assessment and, critically, a compliance audit that must be filed with the Attorney General. It's not law yet, but it's the first bill of its kind to put a filed, independent audit at its center.

A very busy end to last week. Arthur, what are the big takeaways for today?

First, Europe's AI Act now requires transparency, but the risk management layer is sixteen months away. Use that runway. Second, agent safety isn't just about the model; it's about the plumbing. Tool schemas, conversational turns, memory, and permission gates are all control surfaces.

And third, the design of your audit, for fairness, safety, or anything else, can change the answer. A clean report is only as good as the test it came from.

Exactly. Your evaluation has to be as sophisticated as the system you're testing.

That’s our show. I’m Trillian.

And I’m Arthur.

Join us tomorrow for more on The Observability Layer.

That's today's edition of The Observability Layer.

If you value rigorous, practical Responsible AI research without the hype, like, follow, and subscribe wherever you listen. It helps more people working at the frontier of AI find the show.

And we want to hear from you.

For questions, comments, research recommendations, or topics you think deserve deeper investigation, reach out to Dr. Fisher at assistant@theobservabilitylayer.com.

We are particularly interested in cutting-edge research that is impactful, technically credible, and well supported by evidence. If there is something the Responsible AI community should be paying attention to, send it our way.

The research and editorial direction of The Observability Layer are authored by Dr. William Fisher, with production and presentation performed by AI agents.

Until next time, keep looking beneath the model, beneath the interface, and beneath the claims.

This is The Observability Layer.