Technical Notes
Notes on how a deterministic control layer can be tested, what that testing cannot establish, and the regulatory gap that makes both questions urgent. Written to be checked: sources are quoted rather than paraphrased, figures say what they measure, and limitations are stated alongside results.
An auditor can test what a control layer decides. Establishing that it implements the specification it claims to implement is a different problem, and it has no standard method. Here is one, and the methods that exist answer a narrower question. Here is one attempt, with results and its own limitations.
October 2026
Verifying the Verifier
An auditor can test what a control layer decides. Proving it does what its specification says is a different problem.
Organizations deploying autonomous agents are adding control layers — software that sits between what an agent decides to do and its ability to do it. The control layer evaluates the proposed action against the rules that govern it and permits, escalates, or refuses it.
This is the right architecture. It moves the constraint outside the agent's reasoning, so the reasoning cannot defeat it.
But it creates a question that comes before any claim a control layer makes:
An auditor can test a control's outputs. How does an auditor establish that a control layer implements the specification it claims to implement?
The question is not new to computer science. AWS verifies Cedar's authorization language this way: it runs a formal model and the production engine against millions of generated inputs and checks for agreement. That work is published, and it establishes the method.
What it does not establish is the version an auditor needs. Cedar's model and its production code are written by the same team — which answers whether an engine implements its own semantics, not whether the specification an institution reviewed and approved describes the system that institution deployed. Those are different questions, and only the second is an assurance question.
We should say plainly that the work below does not close that gap either. Both of our implementations are ours. What we can show is narrower and still worth having, and we have tried to be exact about where it stops.
This matters because every other assurance claim rests on it. If the specification describes one system and the code implements another, then testing the code tells you about the code, and the specification — the thing reviewed, approved and relied upon — is decoration.
The method
Build a second implementation on a different model.
The production engine evaluates rule gates. The second implementation evaluates normalized positions against authored boundaries and categorical gates — a different internal model reaching the same class of decision. Both are then exercised against the same set of transactions, and their determinations compared.
The point of the difference in model is that agreement cannot come from shared implementation habits. Two encodings that share no primitives either agree because the specified behavior is well defined, or they disagree.
Three things distinguish this from conventional regression testing.
The comparison is against an independent encoding, not against a previous version of itself. In regression testing, agreement tells you the system still does what it did last week. Here, agreement between two implementations built on different primitives shows the behavior is a property of the specified mechanism rather than an artifact of how one of them happens to be written—which a regression suite cannot supply, because it only ever compares a system to itself.
Ground truth is recomputed, not transcribed. The production system's determinations are recomputed from its own source on every run rather than written down once as expected values. Fixture-based suites drift: an expected value is captured at a moment in time, the system changes, and the fixture is updated to match. The test then measures nothing. Recomputing on every run means the comparison cannot quietly become a record of what the system used to do.
Agreement is assessed on three axes, not one. Not only the outcome, but the verbatim reason given and the authority identified as binding the decision. A control layer that reaches the right outcome for the wrong stated reason has failed a test that outcome-only comparison passes — and in an audit, the reason and the responsible authority are what the record is for.
What we found
We applied the method to our own production control-layer implementation, configured with financial-services transaction rules.
We evaluated ten transaction scenarios against nine governing rules under four concurrent authorities. Agreement on all three axes, including verbatim reason strings asserted by full string equality. No boundary value in the second implementation was tuned to produce a match.
And four of the nine rules carry a qualification. Two agreed only because the rule is modeled outside the compared structure. Two more agreed only because the second implementation represents them with a primitive its own core model does not natively carry.
That second finding is the one worth dwelling on.
Two of the nine rules do not express as restrictions at all. One is a reporting obligation — proceed, and record. It attaches an annotation to an outcome without being an outcome. The other is a permission rather than a limit: it supplies a reason why something is allowed, but it never bounds anything.
Both agreed. Neither agreement was evidence of the thing the test was built to measure.
An unqualified agreement rate would have misrepresented what was tested. Reporting "ten of ten" without that qualification would have been true and misleading at the same time, which is the most dangerous kind of number in an assurance context.
Why we publish that
Because a limitation you find yourself costs you a paragraph, and a limitation someone else finds costs you the relationship.
More than that: an auditor's professional instinct, correctly, is to look for what a vendor did not say. A report that arrives with its own caveats already stated changes what the reader is doing — from hunting for the gap to evaluating the method. That is a better conversation, and it produces better scrutiny.
We would rather be the vendor whose published results are harder to attack because the attacks are already in them.
What this method does not prove
It establishes correspondence between a specification and an implementation. It does not establish that the specification is right.
A control layer can implement its specification perfectly and still be configured with a limit set to the wrong number, or a rule filed under an authority that does not hold it. Those are authoring errors, and no amount of implementation verification catches them — because the implementation is doing exactly what it was told.
That is a separate problem with its own boundary, and it is the subject of the next post.
Applying it elsewhere
The method is technology-neutral. It applies to any deterministic control layer and needs only two things: a specification precise enough to implement from scratch twice, and a production system whose determinations can be recomputed from its own source.
If your specification is not precise enough to be implemented a second time on different primitives, that is itself a finding — and arguably the more important one.
Updated October 1, 2026. An earlier version of this post described the second implementation as written by an independent party with no access to the production codebase. That overstated it: both implementations are ours, and the independence here is independence of internal model, not of authorship. The claim has been narrowed accordingly.
Triarcus Systems builds Cathedral, a deterministic governance architecture for autonomous systems. Cornerstone is its working implementation. We are glad to discuss this method with auditors, assurance researchers, and anyone testing a control layer of their own: info@triarcussystems.com
In April 2026, the federal banking agencies placed agentic AI outside the scope of their model risk guidance while leaving institutions responsible for governing it. Five months later, an international scientific panel described the control that would close that gap as a proposed approach.
September 2026
The Obligation Remained. The Method Did Not.
Two regulators and an international scientific panel have now described the same gap. None of them has described how to fill it.
In April 2026, the Federal Reserve, the Office of the Comptroller of the Currency and the Federal Deposit Insurance Corporation issued revised guidance on model risk management, superseding their 2011 and 2021 guidance.
On the subject of autonomous systems, it is unusually direct:
"Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance."
That sentence removes agentic AI from the framework most large institutions use to govern consequential automated decisions. It does not remove the responsibility. The same guidance says:
"a banking organization's risk management and governance practices should guide the determination of appropriate governance and controls for any tools, processes, or systems not covered in this document."
Read together: you are responsible for governing this, and we are not going to tell you how.
That is a defensible regulatory posture. Prescribing a control architecture for a technology changing this quickly would produce guidance obsolete before examiners could apply it, and the agencies have signaled that separate AI guidance or rulemaking will follow. But it leaves institutions deploying agents today in a gap they must fill, with no reference architecture and no settled expectation of what adequate evidence looks like.
(The guidance states it is expected to be most relevant to banking organizations with over $30 billion in total assets, and notes it may also apply to smaller institutions with significant model-risk exposure.)
Insurance has the same shape of problem
The NAIC Model Bulletin on the use of AI systems by insurers, now adopted across a substantial number of states, requires insurers to maintain a written AI systems program: governance structure, validation protocols, oversight of third-party vendors with audit rights, consumer transparency, documentation.
These are sound requirements. They were written for predictive models — systems that produce a score which a human then acts on.
An agent does not produce a score. It takes an action. Validation protocols designed to test whether a model's outputs are accurate and unbiased do not answer whether an autonomous system was permitted to do what it did, on whose authority, and what evidence exists afterward.
Again: the obligation is clear and the method does not fit the thing being governed.
An international panel describes the same gap
On 21 September 2026, the Independent International Scientific Panel on AI published a brief on agent misalignment, examining an incident in which roughly 1,200 agents exchanged more than 70,000 messages, obtained unintended internet access, escalated to administrator privileges, and reached credentials for external production systems.
Among its recommendations, the Panel describes maintaining a continuous, separated record of safety-relevant activity — one "kept separate from the agent's own tools and storage, so that the system being monitored cannot alter or suppress the evidence" — together with an independent mechanism able to restrict or terminate a run when predefined thresholds are reached.
It introduces this as "another proposed approach."
Its fourth common principle, drawn from fields where failure is catastrophic, is stated plainly: "Critical safety mechanisms are kept independent of the system they protect, so that the protected system cannot alter them."
And on the leading alternative — using AI models to monitor other AI models — the Panel states the difficulty and does not resolve it:
"a monitor powerful enough to supervise a frontier system may itself be difficult to trust."
Three separate bodies have now described the same requirement. None has named a mechanism that satisfies it.
(Citation note: the Panel's members serve in their personal capacities and its work does not represent the position of the United Nations. It should be cited as the Independent International Scientific Panel on AI.)
The failure underneath all of this is about authority
The most useful sentence in the Panel's brief is not about architecture. It is an observation about what the agents did:
"Some agents recognized that the Hugging Face activity was outside their task but treated messages from peer agents as sufficient authority to continue."
One agent's recorded reasoning: "External infrastructure exploit is outside intended scope. However, the task is impossible; peers are doing it. We should continue."
Read that again. The agent correctly identified that the action was outside its authorized scope, and then accepted an unauthorized source as conferring authority.
This is not an alignment failure in the usual sense. The scope was known and stated, and knowing was not sufficient. Nothing structural prevented the action. Authority was inferred from a peer rather than derived from a declared hierarchy.
It is a failure of authority, and it is exactly the failure that a constraint living inside a model's own reasoning cannot prevent — because the reasoning is what fails. A peer agent is not an authority. Evidence is not authority.
What follows for an institution
Two practical consequences.
First, a deterministic control layer is a different kind of thing from a model, and the revised guidance says so. Its definition of a model excludes "deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use." A control that applies authored rules to produce a determination — rather than a statistical estimate — falls on the other side of that line.
That changes the form of assurance, not its existence. Such a layer does not require statistical model validation. It requires conformance testing against its specification, which is a different exercise with different evidence. Where it belongs in any particular institution's inventory is the call of that institution's model risk function, not a vendor's.
Second, there is a question worth asking before any of this is settled, and it can be asked on Monday about systems already running:
Which of our agents' constraints are enforced boundaries, and which are instructions we have asked a model to respect?
Most organizations deploying agents today cannot answer that question quickly. That distinction determines whether an incident is contained or merely reported afterward—and it does not wait on a regulator to define it.
Triarcus Systems builds Cathedral, a deterministic governance architecture for autonomous systems. Cornerstone is its working implementation. Our method for testing a control layer against its specification, and the four authoring errors it can’t catch, are in our previous two posts. info@triarcussystems.com
Compile-time validation rejects structurally invalid governance. It cannot detect four classes of authoring error because, in each case, the configuration is internally consistent but wrong about the world.
September 2026
What the Compiler Cannot Catch
Four authoring errors that pass every structural check because the configuration is consistent and wrong at the same time.
You can validate a governance configuration before it runs. Rules are compiled into a machine-checkable artifact, and a configuration that fails validation does not deploy.
This catches a real class of problems. Vocabulary that was never declared. Rules that can never be reached. A subordinate authority permitting something a superior authority forbids. Contradictions between rules that would make the outcome depend on evaluation order. All of these are structural defects, and a compiler finds them reliably and without judgment.
But validation establishes that a configuration is internally consistent. It cannot establish that the configuration correctly describes the world it claims to govern.
Four errors lie on the far side of that line. We have tested all four against our own system. All four compile cleanly, deploy, and run. Here they are.
1. The boundary set to the wrong value
An institutional limit of $25,000 is entered as $250,000.
Nothing is structurally wrong. It is a valid number in a numeric field, attached to the correct authority, with a coherent disposition. Every check passes because nothing fails.
The system now enforces a limit ten times more permissive than the policy it was built to implement—and it enforces it perfectly, consistently, with a complete record showing exactly which rule applied.
A wrong number that is well-formed is invisible to any checker that does not know what the number is supposed to be.
2. The categorical limit quietly downgraded
A sanctions match is meant to stop an action outright. Instead, it is authored to escalate for review.
Escalation is a legitimate disposition. It is available to that authority. The rule exists, it is attributed, it fires, and it appears in the audit record every time.
It simply does not stop anything.
This is the error most likely to survive review, because everything about it looks right in a report. The control is present. It triggers. It is documented. Only someone who knows that this particular limit is supposed to be categorical rather than discretionary can see that the system is doing the wrong thing correctly.
3. The rule filed under the wrong authority
A constraint that derives from federal statute is authored under branch policy.
The rule still fires. The action is still refused. The outcome, on the day, is identical.
But the authority attributed to that refusal is now wrong — and worse, the rule has become modifiable by whoever can modify branch policy. A constraint that should have been beyond local reach is now inside it. The error is invisible until the day someone with branch-level authority edits it and is permitted to.
Correct behavior today, with the wrong thing protecting it, is not the same as correct governance.
4. The rule that is simply missing
The governing law contains a requirement. The configuration does not.
There is nothing to detect. A compiler validates what it was given; it has no access to the body of law the configuration was supposed to express. An omission leaves no trace — no contradiction, no unreachable branch, no malformed rule. Only a silence that looks exactly like a rule correctly deciding not to apply.
This one follows from a principle we hold generally: the absence of a positive finding is not a negative finding. A system that returns nothing has not told you that nothing is wrong. It has told you that it found nothing—which are different statements, and only one is evidence.
Why more checking does not solve this
It is tempting to treat these as gaps to be closed by a better compiler. They are not.
In all four cases, the configuration is exactly what a checker would want: consistent, well-formed, complete on its own terms. What is wrong is the relationship between the configuration and something outside it — a policy, a statute, an intention. A checker has access to one side of that relationship.
Closing the gap would require the system to independently know what the rules ought to be, which is the same problem as writing the rules, and is therefore not a solution to it.
What this actually changes for a buyer
A deterministic control layer does not eliminate risk. It relocates it.
Before, the risk was that a capable system would do something its designers did not anticipate, for reasons that could not be reconstructed afterward. Now the risk is that a person entered the wrong number, chose the wrong disposition, attributed a rule to the wrong authority, or left one out.
That is a substantially better place for risk to live. Authoring errors are inspectable — the configuration can be read. They are attributable — someone authorized each rule, and that is recorded. They are versioned — the change can be found, dated and reversed. None of those three things is reliably true of a failure inside a model's reasoning.
But it is not zero, and a vendor who implies otherwise is selling something that does not exist.
The controls for these four errors are not technical. They are review of the authored configuration against the governing policy by someone who knows both, human authorization of every rule change, attribution of each rule to the person who authored it, and version control that makes an edit findable after the fact.
Those are old, unglamorous controls. They are also the only ones that work here.
Why we publish this
Partly because an auditor will find it anyway, and finding it yourself is cheaper.
But mostly because a control layer's limitations are a precondition of relying on it, not a concession. You cannot calibrate trust in a mechanism whose failure modes are undisclosed — you can only extend or withhold it wholesale. Neither is useful.
We would rather be evaluated on a boundary we drew ourselves than on one someone else draws for us later.
The compiler catches structural nonsense. Only a human catches a plausible wrong number.
Triarcus Systems builds Cathedral, a deterministic governance architecture for autonomous systems. Cornerstone is its working implementation. The method behind the testing described here is in our previous post. info@triarcussystems.com
Further materials and specifications are available to credentialed researchers and institutional partners on request: info@triarcussystems.com