Quick take
AI privacy is plumbing, not policy. Map every data flow. Minimize what you send to models. Control who can replay prompts and access logs. Set retention rules that are actually enforced. Do sensitive work locally and pass reduced representations upstream. If you treat privacy as a late-stage review, you’ll fail the audit.
My background in NATO Cyber Defense taught me something that most engineers learn too late: data classification isn’t a theoretical exercise. When you’re operating in an environment where information leakage has consequences beyond a compliance fine, you develop a different relationship with data flows. You map them. You minimize them. You assume every copy of data is a liability until proven otherwise.
That mindset transfers directly to AI systems.
The Problem Nobody Maps
Most AI features touch far more data than the visible prompt. In a typical RAG workflow, the user submits a query, your system retrieves context from a knowledge base, the model receives both the query and retrieved documents, it generates a response, and that response gets logged for quality monitoring.
At each step, data is copied. The user’s query is in your application logs, in the retrieval system’s query log, in the model provider’s request log, in your quality monitoring dashboard. The retrieved documents – which might contain sensitive customer data – now exist in your model provider’s system too, subject to their retention policy, not yours.
If you can’t draw this flow on a whiteboard in under two minutes, your privacy controls are guesswork. I start every privacy review by asking the team to map the flow. Most teams can’t do it. That’s the first problem to fix.
Minimize Before You Send
Data minimization is the single most effective privacy control in AI systems. Not because it’s elegant, but because it reduces blast radius. Data you don’t send can’t be leaked, retained, or trained on.
Practical minimization looks like this:
Strip identifiers early. Before the prompt is assembled, remove names, emails, account IDs – anything that isn’t required for the model to produce a useful response. If the model needs to reference a user, use an opaque session token that maps to the real identity only in your system.
Send summaries, not documents. If you need context from a 20-page contract, summarize the relevant section locally and send the summary. The model doesn’t need the full document. Your privacy exposure drops by an order of magnitude.
Separate sensitive from useful. Not all data carries the same risk. Split your workflows so that high-sensitivity data – medical records, financial details, authentication tokens – is processed locally with stronger controls. Lower-risk data can flow through standard AI paths. This tiering reduces the scope of every privacy review and makes incident response simpler.
Local First for the Dangerous Bits
Some operations should never leave your infrastructure. PII detection, redaction, and sensitive-content classification should run locally, on models you control, before anything touches an external API.
The pattern is straightforward: do sensitive work where the data already lives, then pass a reduced representation to the cloud model. This isn’t about avoiding cloud AI entirely. It’s about being deliberate about what crosses the boundary.
I’ve helped design pipelines where the first stage runs a local model to detect and redact PII, the second stage sends the sanitized content to a cloud model for the actual task, and the third stage re-attaches the redacted information only in the final response shown to the authorized user. The cloud model never sees real PII. The logs never contain it. The attack surface shrinks dramatically.
Logs Are the Quiet Privacy Gap
AI features generate logs that teams don’t think about. Prompt logs for debugging. Response logs for quality monitoring. Replay tools for incident investigation. Evaluation datasets built from production traffic.
Each of these creates a copy of user data that lives outside your normal data governance. And because these are “internal tools,” they often have broader access than production databases do.
Lock them down the same way you lock down production data:
- Access control. Not everyone who can view the dashboard should be able to replay prompts containing user data. Restrict access by role and audit who accesses what.
- Retention limits. Prompt logs don’t need to live forever. Set a retention window – 30 days is plenty for most debugging needs – and enforce automatic deletion.
- Audit trails. Know who accessed which logs and when. This isn’t optional for regulated industries. It shouldn’t be optional for anyone.
Vendor Questions That Actually Matter
When evaluating AI providers, skip the marketing page and ask these questions directly:
- Is customer data used to train or improve models by default? How do you opt out, and is the opt-out verified?
- What data is retained after a request completes? For how long? For what purpose?
- Where does processing happen geographically? Who on the vendor’s side can access request logs?
- How are deletion requests handled? What’s the SLA? Is deletion cryptographic or simply a database flag?
Write the answers down. Put them in your vendor assessment. Revisit them annually, because vendor policies change without notice.
Governance That Survives Audits
Heavy governance processes don’t survive contact with reality. Teams skip them, shortcuts accumulate, and the audit reveals a gap between policy and practice.
Keep governance light and concrete:
- One data flow map per AI feature. Inputs, retrieval sources, logs, outputs, retention. Fits on a single page.
- A documented purpose for each data category. Why is this data in the pipeline? If you can’t answer, remove it.
- Tested deletion paths. Not “we have a process for deletion.” Actually run it. Verify the data is gone. Do this quarterly.
Privacy is a design constraint, not a compliance checkbox. Build it into your AI pipeline the same way you build in authentication and authorization: as infrastructure that runs automatically, not as a review that happens after the fact.
Security, stability, performance – in that order. Privacy falls under security. It goes first.