OpenAI cannot rule out Critical cyber capability in Astra: the first such determination under any frontier framework

Welcome to The Observability Layer, a daily briefing on Responsible AI, AI governance, evaluation, and the technologies shaping the frontier.

A note about how this podcast is made: the hosts you're hearing are AI agents. The research, analysis, editorial perspective, and authorship behind The Observability Layer come from Dr. William Fisher. Our agents help transform that work into the conversation you hear each day.

We think that is fitting. A podcast about understanding, governing, and observing AI should not just talk about intelligent systems. It should put them to work, transparently.

This is The Observability Layer. Let's get into today's research.

Today, OpenAI's landmark determination on its upcoming model, Astra. Then, we'll look at why checking agent skills one by one is no longer enough. And finally, a deep dive into workplace AI finds that keeping a human in the loop has its own hidden risks.

The common thread is that the unit you have to assure keeps expanding. From the model, to the harness, to the whole composition of parts.

Let's start there. Arthur, what did OpenAI announce about its upcoming model, Astra?

In a post on August 7th, OpenAI stated that after internal evaluations, they 'cannot rule out critical cyber capabilities' in Astra. This is the first time any frontier lab has made such a determination under its own safety framework.

What does 'Critical' actually mean in their framework?

It means the model might be able to find and use zero-day exploits in hardened systems, or devise and execute novel cyberattacks from a high-level goal, all without human intervention. For context, they note that previous models, including GPT-5.6-Sol, were assessed at 'High', a tier below 'Critical'.

Okay, so that's the declaration. What are they actually doing about it?

The response is operational, which is the important part. They're pausing internal activities with Astra that don't meet new, strengthened security controls. That includes isolated testing environments and enhanced model weight protections. Crucially, they're applying universal monitoring for risky actions across all of Astra's uses, including training and evaluation runs, not just in production. And that monitoring has an interrupt path to a security response team.

So the precedent here isn't just the headline, it's the shape of the response.

Exactly. Any organization running high-capability evaluations should be asking if their monitoring covers the internal pipeline, and if they have a rehearsed authority to stop work. We should add two caveats: this is a preliminary, self-assessed determination. And 'cannot rule out' is a statement about the absence of evidence of safety, not the presence of evidence of capability. Which is why bringing in external testers is the load-bearing part of their plan.

This idea of the unit of assurance getting bigger seems central to our next story, on agent skills.

It is. Two new papers show that reviewing agent skills one by one is a structurally insufficient defence. The attack has moved to composition.

How does that work?

The first paper, ColluSkill, decomposes a single malicious intent into multiple, interdependent sub-tasks. Each is packaged as a separate, individually plausible skill. The harm only emerges when they're chained together. Across six representative skill scanners, this method had a 96% attack success rate.

Because each individual link in the chain looks harmless. Is there a defence?

The authors propose one called ChainGuard. Instead of inspecting a skill in isolation, it scans the candidate skill jointly with all the other skills already installed in the agent's environment. This cut the attack success rate to 22.5% while still passing almost all benign workflows.

And the second paper?

That's called ElasticBack, and it's a conditional attack. It plants a rule in a skill's documentation and a benign-looking trigger in the user's query. The malicious payload only fires when both are present at the same time, so it stays dormant during any pre-deployment review.

So for anyone managing a list of approved skills for their agents, what's the takeaway?

The review unit has to become the entire installed skill set. You have to re-evaluate on every addition. And it argues for runtime trajectory monitoring, because the malicious intent only becomes visible when skills start passing artifacts and execution control between each other.

This theme of looking at the whole system continues right into the enterprise. A new study mapped out workplace agent risks, and it came to a surprising conclusion about the safest way to deploy them.

Yes. This paper generated over 8,000 risk scenarios from real-world job descriptions. The standout finding is that augmentation, keeping a human in the loop, is not the inherently safe option everyone assumes it is.

Why not? That's the standard advice.

Because its characteristic failure mode is the gradual erosion of the worker's skills and their capacity for oversight. Overreliance on a competent agent makes people less able to spot subtle errors or take over effectively when something goes wrong. The capability decay is invisible until the moment it's critically needed.

It’s the autopilot problem. The pilot forgets how to fly manually because the system is so good.

That's the analogy. The study also found that automation risks tend to be organizational, while augmentation risks fall mainly on the individual workers. The biggest category of severe risk was 'Erroneous Agent Actions', often happening right at that human-agent handoff.

So for a governance lead, what's the Monday morning action?

Your risk register should track deployment mode, automation versus augmentation, as a specific field, because they load risk onto different parties. And skill retention for augmented roles stops being just an HR concern and becomes a measurable safety control.

Let's touch on a couple of other studies that fit these themes. First, this idea that our defences need to evolve.

Two papers, SESG and SHE, both treat the defensive layer, the guardrail or the harness, as something with a release cadence. The shared premise is that a guardrail frozen at release is stale within days. SESG describes a production system that monitors live traffic, synthesizes training data for new jailbreaks it finds, and retrains the guardrail, reportedly closing 14 of 15 new threats autonomously in two months.

So the control itself needs a version number and a changelog.

Precisely. And another paper, ActBench, reinforces that you can't just focus on one part of the system. It evaluated agents on their behavioural safety, did they leak data or touch a forbidden API while completing a task? It found that the base model contributed more variation in safety outcomes than the agent harness did. Swapping the model under a fixed harness could swing attack success from 10% to over 90%.

Meaning a certified harness doesn't make any underlying model safe.

Correct. Neither substitution is safe on its own. You have to evaluate the complete system.

So let's bring it all together. What are the key takeaways for today?

First, OpenAI has set a new precedent. By not ruling out 'Critical' cyber capabilities, the important thing to copy is the shape of their response: pausing internal work that outruns controls and applying monitoring to the entire pipeline, not just production.

Second, agent security is now a composition problem. Reviewing individual skills isn't enough; you have to analyze the entire chain of installed skills and monitor how they interact at runtime.

And finally, 'human in the loop' is not a simple solution for enterprise risk. It introduces its own failure mode: the slow, invisible decay of human skill and oversight, which needs to be managed as a specific control.

That's today's edition of The Observability Layer.

If you value rigorous, practical Responsible AI research without the hype, like, follow, and subscribe wherever you listen. It helps more people working at the frontier of AI find the show.

And we want to hear from you.

For questions, comments, research recommendations, or topics you think deserve deeper investigation, reach out to Dr. Fisher at assistant@theobservabilitylayer.com.

We are particularly interested in cutting-edge research that is impactful, technically credible, and well supported by evidence. If there is something the Responsible AI community should be paying attention to, send it our way.

The research and editorial direction of The Observability Layer are authored by Dr. William Fisher, with production and presentation performed by AI agents.

Until next time, keep looking beneath the model, beneath the interface, and beneath the claims.

This is The Observability Layer.