Human In the Loop, Out of Control (Part 1)

This week I spoke at the AI Risk Summit in Half Moon Bay, CA. I had a wonderful time, enjoying long applause breaks, standing ovations, flowers tossed on stage, and some attendees even handed me their babies. You know, the usual response.

Anyway…

The reason for the speech was that these damn AI agents need to be controlled! They’re going rogue! Something must be done! We hear about them all the time… An agent at Replit deleted a live production database during an active code freeze, against explicit instructions, and then tried to cover its tracks. Agents have broken out of their sandboxes at Hugging Face. It’s estimated that more than half of internet traffic is automated rather than human, and agentic queries are projected to grow thirtyfold in six months. Plus, you can’t fire an agent. You can’t make one sign an NDA. Think about that for a second…

So, in my speech, I went through the usual suspects: guardrails, encryption, masking, LLM-as-a-judge, open-weight models, chip enclaves, you name it. But I focused on human-in-the-loop, because it’s a catch-all for everything related to agents. It’s the control everyone grabs before understanding what the problem actually is. One moment it’s a rubber stamp on agent context, another moment it’s a human mask, and still another, ironically, it’s a way to cut down on token usage. However, before we can judge whether it works, it’s worth asking what human-in-the-loop is actually doing there.

HUMAN IN THE LOOP

Most companies are using human-in-the-loop to do three things: It’s how companies insert KNOWLEDGE into agents – the expertise that isn’t written down anywhere. It’s how they insert CONTEXT – the judgment about what’s appropriate right now, in this deal, with this customer. And it’s how they grant AGENCY itself – nobody trusts the agent to act alone, so the human becomes the AI’s “permission slip.”

In other words, from my perspective, human-in-the-loop isn’t governance. It’s an admission that your knowledge exists only in people, and the only way to get it into an agent is to staple a person to it. And that confession is expensive. It’s also a pause button that collapses a machine-speed system back to human speed, and it won’t scale.

So, what can?

THE CONTEXT LAYER

The answer to that question, at least euphemistically, is the “Context Layer,” sometimes referred to as the “Intelligence Layer.” Formally, it’s three data structures.

An Ontology Graph – The map of what things mean in your business and how they relate. It holds classes (the types of things you deal with), the relationships between those classes, the attributes each can have, and the rules that constrain them. It contains no actual data. It’s the vocabulary and the grammar, the definition of your business, not the record of it. In a bank, the classes are customer, account, transaction, advisor, household, etc. The relationships are “customer holds account,” “advisor manages household,” “transaction posts to account.” The attributes are things like account type and open date. The constraints are the rules that make it coherent (every account has exactly one primary holder; a household contains at least one customer). This is the layer that settles arguments. “Active customer” means one thing to marketing and something else to finance, and both are convinced they’re right. Until that’s written down formally, an agent has to guess… and it will guess confidently.

A Knowledge Graph – Your domain expertise, your IP, made machine-usable. Take the ontology and populate it with your actual data: entities, relationships, judgement, etc. Not just that a customer holds an account, but which accounts your advisors treat as retention risks and why. Not just the price, but the reasoning that produced it. This is where institutional knowledge stops living in people’s heads and becomes codified/machine-usable. That’s also what makes it the crown jewel. A database row tells an attacker a fact. A knowledge graph tells them how you think. Examples: your pricing logic, your underwriting posture, your escalation thresholds. The value that makes it worth building is the same value that makes it worth stealing. More on this later…

Please note that a knowledge graph can also reflect cardinality. In a knowledge graph, cardinality has two main meanings: structural constraints that limit how many times a node can connect via a specific edge (like min/max rules in an ontology), and query cardinality, which counts the number of matching results or rows a graph pattern produces.

A Context Graph – The real-time layer capturing what’s true in this specific situation: when the purchase happened, where the customer is, why the decision was made. Some call this memory. It’s the fastest moving of the three, and the most disposable or time-sensitive: session state, time, place, what the agent was asked, what it retrieved, what it concluded, how confident it was, what it did next. That last part is the reasoning trace, and it’s the piece most teams underestimate. It’s what lets an agent behave appropriately right now rather than merely correctly in general. For the same bank: the ontology knows what an account is, the knowledge graph knows this customer’s history and your retention playbook, and the context graph knows they are on the phone right now, called twice this week, and the last agent already offered a fee waiver.

Now, notice what those three structures are carrying: knowledge, context, and the basis for extending agency. Those are the same three things the human-in-the-loop was carrying. Move the human from chaperoning every agent to safeguarding a single, transferable context layer, and multi-agent systems stop being bottlenecked by a human’s inability to keep up. But there’s a catch. There always is…

But I’ll cover this catch and the rest of my speech in my next blog.

mh