Quick take
AI compliance is a design problem, not a paperwork problem. Build a data inventory, a model registry, and audit logging before you ship – not after legal gets involved. The organizations shipping fastest are the ones that treat compliance as architecture, not bureaucracy.
My perspective on AI compliance is shaped by two things: working on AI adoption at large enterprises and my work with NATO on cyber defense. Those are very different worlds, but they share one uncomfortable truth – organizations that treat security and compliance as an afterthought tend to have the worst incidents and the slowest response times.
In the defense world, you learn quickly that compliance isn’t about checking boxes. It’s about building systems that can answer hard questions fast. Where did this data come from? Who authorized this action? What changed between yesterday and today? When something goes wrong at 2 AM, nobody cares about your compliance document. They care about whether your systems can provide answers.
That same principle applies to enterprise AI. Just with lower stakes and, unfortunately, less discipline.
The questions that actually matter
I’ve sat through dozens of compliance reviews for AI systems. They all converge on the same handful of questions:
- Where does user data go during inference, and is any of it retained?
- Can you trace a specific output back to the model version and prompt that produced it?
- How do you detect and handle unsafe, biased, or hallucinated outputs?
- Who approved this use case, and what risk assessment was done?
- If the model provider changes their terms or has a breach, what’s your exit plan?
If your engineering team can’t answer these within minutes, you aren’t ready for production. Full stop. I’ve seen AI projects delayed six months because the team couldn’t explain their data flow to a procurement review. That isn’t a compliance problem. That’s a design problem.
Data governance is the foundation
Start with a data inventory. Not a theoretical one – a real, maintained list of what data enters your AI pipeline, how it’s classified, where it’s processed, and when it’s deleted.
This sounds basic. It is. Most teams still skip it because it’s boring. Then, three months in, they discover their LLM provider’s terms allow training on API inputs, and they’ve been sending customer PII through an endpoint with no data processing agreement.
From my NATO experience: you don’t get to decide what data classification matters after the incident. You decide before. The same applies here. Know your data flows. Classify them. Enforce the policies technically, not just on paper.
Model accountability isn’t optional
You need a model registry. Every inference in production should be traceable to a specific model version, a specific prompt version, and a specific configuration. This isn’t overengineering. This is the minimum bar for debugging, incident response, and regulatory compliance.
What to log for each request:
- A stable request ID
- Model identifier and version
- Prompt template version
- A hash or summary of the output (not the raw output if it contains sensitive content)
- Timestamp, user context, and latency
In the defense space, we call this “chain of custody for decisions.” In enterprise AI, it’s just good engineering. I’m still surprised by how many teams ship without it.
Human oversight that actually works
The compliance frameworks I’ve seen fail are the ones that require human approval for everything. That doesn’t scale. It creates bottlenecks, and people start rubber-stamping just to keep velocity.
Better approach: tier your use cases by risk.
Low risk (internal tools, human-reviewed outputs): self-service approval, lightweight monitoring. A team lead signs off and you move on.
Medium risk (customer-facing, influences decisions): security review, data assessment, defined rollback plan. One meeting, not a committee.
High risk (financial, medical, safety-critical): full review cycle with legal, security, and domain experts. No shortcuts, but a defined timeline.
The goal is to make the approval path proportional to the risk. Low-risk use cases should ship in days, not weeks. High-risk use cases should have rigor, not paralysis.
Vendor risk is your risk
Every AI provider you use is a critical dependency. Treat it that way. I’ve reviewed vendor contracts where data-handling terms were buried in an appendix nobody on the engineering team had read.
Key questions for any AI vendor:
- Is customer data used for model training? Can you opt out?
- What’s the breach notification timeline?
- What happens to your data if you terminate the contract?
- Can you run the same workload on a different provider if needed?
Lock-in is a compliance risk. If your only option is one provider and they change their terms or have a major incident, you need a plan B that doesn’t require rewriting your entire pipeline.
Three artifacts you actually need
Forget the 50-page compliance documents. Maintain three living artifacts:
- System card. One page per AI system: what it does, what data it touches, known limitations, risk tier, and owner.
- Data inventory. Where data comes from, where it goes, classification, retention, and deletion procedures.
- Model registry. Model versions in production, evaluation results, prompt versions, and deployment history.
Keep them in version control, not in a shared drive nobody checks. Review them quarterly, or whenever the model or data pipeline changes.
The real competitive advantage
The enterprises shipping AI fastest right now aren’t the ones ignoring compliance. They’re the ones that built it into their architecture early, kept it lightweight, and made it a development practice instead of a legal review.
Compliance built into the system is invisible. Compliance bolted on afterward is a project that never ends.