Governance is not the fifth layer

There is a useful frame doing the rounds. On the NVIDIA AI Podcast, Red Hat CTO Chris Wright and NVIDIA’s Justin Boitano describe the enterprise AI factory as a five-layer cake: accelerated hardware, hybrid cloud infrastructure, models, agents, and production-grade governance. It is a good frame. It is the first vendor articulation I have seen that treats enterprise AI as an infrastructure problem rather than a model problem, and it is right about the thing most organisations get wrong — that you cannot pilot your way to production.

I want to take it seriously enough to disagree with one part of it.

First, why the stack exists at all

The economics are the reason this is urgent rather than interesting. Red Hat’s published position is that the unit cost of a token is falling roughly 75 to 90 percent a year, while enterprise token consumption is rising more than 500 percent a year. Their own projection has total consumption rising twenty-four-fold by 2030.

Read those together and the conclusion is uncomfortable: falling prices will not save you. Consumption is outrunning deflation, which means efficiency stops being a procurement question and becomes an architectural one. That is the real argument for having a stack at all — not elegance, but the fact that an unstructured estate gets expensive faster than the market gets cheap.

The amendment

Governance is not a layer. It is a property of all five, and the organisations that treat it as the top of the stack are the ones whose pilots never ship.

I say this from a specific vantage point. My work spans defence and a Big Five Canadian bank. In both, the question that stops a promising AI system is almost never “does it work.” It is “can you evidence what it did, on what data, under whose authority — and can you show the control existed before the incident rather than after it.” That question cannot be answered at the top of the stack, because the evidence it requires is generated further down, or it is not generated at all.

Here is what governance looks like at each layer if you build it in rather than bolt it on.

Where the machine physically sits determines which country’s law reaches the workload running on it. That is not a legal footnote; it is an architectural constraint that is expensive to reverse. For a federally regulated institution it drives third-party and concentration risk. For defence and government it sets the accreditation boundary — and that boundary starts at the metal, not at the application.

The hardware layer is also where confidentiality from the operator is won or lost. Extending trusted execution environments from the CPU to the accelerator is what lets you run a sensitive workload on infrastructure you do not exclusively control. If that is not designed in at layer one, no amount of policy at layer five recovers it.

The economic case for this layer is utilisation: queueing and sharing GPUs across teams so they are not idle. That is correct, and it quietly creates a multi-tenancy problem. Once teams share accelerators you have to answer whose workload can pre-empt whose, what one tenant can observe of another, and what happens to a regulated workload when an experimental one saturates the cluster.

This is also where configuration drift lives. Drift is usually filed as an operations annoyance. In a regulated estate it is a governance failure with a precise definition: the growing gap between the environment you had accredited and the one that is actually running. Every week that gap widens, your assurance evidence describes a system that no longer exists.

Model sprawl is the governance problem of this layer, and it is structural rather than cultural. Capable open models are released weekly, and the distance between a team trying one and a team depending on one is very short. Shadow AI is not people being reckless. It is the predictable outcome when the sanctioned path is slower than the unsanctioned one.

This is why curated model catalogues exist — not as a convenience, but because provenance is a control. For Canadian financial institutions it stops being optional shortly: OSFI’s Guideline E-23 takes effect on 1 May 2027 and expects an enterprise-wide model inventory with lifecycle oversight proportionate to risk. An inventory is not a document you assemble the quarter before. It is a by-product of a serving layer that makes registration the path of least resistance.

This is the least solved layer and the one I would watch most closely. An agent is a non-human actor that takes consequential action, and almost every enterprise identity model was built on the assumption that actions trace to a person. An agent needs three things we are not yet good at issuing: an identity of its own, an authorisation scope narrower than the human who launched it, and an execution trace detailed enough to reconstruct a decision after the fact.

The industry is converging on real answers — workload identity frameworks such as SPIFFE and SPIRE, kernel-isolated sandboxes, standardised tool interfaces. What has not caught up is the accountability question underneath. When an agent takes an action that turns out to be wrong, the organisation must be able to say who authorised it to act, within what limits, and on whose behalf. That is an architecture problem long before it is a policy problem.

The top layer is real and it matters. It is the assurance surface: the inventory, the monitoring, the thresholds, the reporting a board or a regulator actually reads. My objection is not that it exists. It is the implication that it is where the work is done.

Everything the fifth layer presents was produced by the four beneath it. The inventory is only as true as the serving layer that populates it. The audit trail is only as complete as the agent runtime that emitted it. The residency attestation is only as good as the decision made in the rack. Build layers one to four without asking what evidence they must produce, and the top of the stack becomes a reporting exercise describing a system you cannot account for.

Why this lands differently in Canada right now

Two things are converging. Guideline E-23 arrives on 1 May 2027 and pulls model risk management out of the quantitative-validation corner and across the whole enterprise model estate, AI included. At the same time, sovereign AI capacity is being built here — HIVE’s BUZZ HPC has announced a 320-megawatt AI data centre in the Greater Toronto Area.

The constraint on that build-out will not be manufacturing, and it will not be power, though both are hard. It will be the buyers — banks, departments, agencies — who cannot put a workload onto new capacity until they can evidence how it is governed. The governance question is not a compliance tax applied at the end of the programme. It is a gating factor on the whole market, and it is decided at every layer of the stack.