Regulated AI workloads on shared GPUs run into a sovereignty wall: health, finance, and legal data legal won't let leave. A tier of demand, and an infrastructure problem.
In AI, data sovereignty reads like a compliance footnote and behaves like a whole tier of demand with nowhere to go.
Disclosure: we're building a venture in this category. Figures below are attributed; we name our own stake where it bears on the argument.
Ask an enterprise where its most valuable AI use cases are, and the list comes back fast: claims data, patient records, transaction histories, contracts under negotiation, case files. Then ask why they aren't running yet, and the answer is almost never the model.
It's that legal won't let the data leave.
"Data sovereignty" gets used loosely, so here's the working version: the requirement that specific data stays in a specific place, under a specific jurisdiction's rules, with a provable record of everything that touched it.
For a European hospital, that can mean patient data never leaving the EU, governed by GDPR, with a written agreement and a documented deletion. For a Canadian financial firm, it can mean personal data handled under PIPEDA, with breach notification inside a fixed window. For a US health system, it's the familiar acronym and a chain of custody an auditor can follow.
None of those requirements care how good your model is. They care where the bytes went, who could see them, and whether you can prove it.
This is the crux of running regulated AI workloads on shared GPUs. The economics of cheap, rented compute come from sharing: many tenants on infrastructure no single one owns. That's also exactly what makes a compliance officer uneasy. Public reporting on the open GPU-rental market doesn't soften it. A widely-cited Hacker News discussion (2023) on a major marketplace put it bluntly, "a host can snoop on your workloads" (Hacker News, 2023). Listed prices on those marketplaces look great. The effective cost of getting an unverified host to behave can run 20 to 40% above the sticker (per our competitive analysis of the open-marketplace tier).
The most valuable AI workloads are the ones legal won't let near a shared machine. That's not an edge case. That's a market with the lights off.
Vendors know this, which is why a sensitive-workload tier keeps appearing: one network steers regulated jobs to "Verified Data Center" nodes and warns against its community tier for SOC 2 or HIPAA data (a leading decentralized GPU network, vendor comparison, Apr 2026); another bolts on a separately-certified "Secure Cloud" partner tier (a major GPU marketplace, 2026). The pattern is telling. The default rented machine is treated as not good enough for the data that matters most, so a special lane gets built beside it.
The root cause is a supply-quality gap we map in the flagship, The sub-1% problem in decentralized compute, and the trust mechanism that helps close it is the subject of Proof you can't fake: how verifiable inference changes who you can rent compute from. Sovereignty is the demand. Verification is part of how you serve it.
Strip it to the components a regulator or auditor would recognize, and the list is short but unforgiving:
Isolation that's structural, not promised. Per-job network boundaries so a workload talks only to the coordinator, not the open internet. This closes the exfiltration path rather than asking you to trust it's closed.
A provable chain of custody. Encryption in transit and at rest, plus a record of what ran where, that survives an audit.
Data residency you can specify. The ability to keep a job inside a jurisdiction, by design, not by hope.
A real data-processing agreement. Under GDPR, the operator of someone else's machine is a sub-processor, which means a signed DPA and a guaranteed deletion, not a verbal assurance.
What's missing from that list is telling: a bigger model. The hard part of regulated AI is the infrastructure underneath it.
No architecture makes the input data magically invisible to the place computing on it; absolute technical confidentiality on third-party hardware is a claim to distrust. We're not the only ones working this problem, either: confidential computing and trusted execution environments already shield data in use, and verifiable inference (covered above) proves the work was done as claimed. Each closes part of the gap, and a sovereignty-grade lane borrows from all of them. The right framing is risk reduction you can document (minimization, contractual control, isolation, transparency), not a promise of perfect secrecy. Any vendor who tells you otherwise is selling the claim, not the capability.
Where does our own venture sit in all this? Griddly is being built toward exactly this demand: per-job isolation, data-handling auditable against GDPR, residency you can specify, so the workloads legal currently blocks have somewhere compliant to run. The model was never the bottleneck. The lane was.
Nothing here is an offer to sell a security or investment advice.
Field notes on venture building, AI, and capital. No spam, unsubscribe anytime.
By subscribing you agree to receive AOS Insights e-mails. We use your address only for this newsletter - see our Privacy Policy.
