AI data privacy is the design and operation of how personal or sensitive data is collected, retrieved, transmitted, used, logged, exposed, retained and deleted across the complete AI system. Effective privacy requires purpose limitation, data minimisation, role-based access, provider review, controlled outputs and evidence that the configured path matches the policy.
Inventory every context surface
Map direct user input, retrieved documents, vector or search indexes, model requests, tool arguments, provider logs, application traces, caches, generated output and write-back to operational systems. Identify who can access each surface and for how long.
This map often reveals that the sensitive path is not the visible chat. A debugging trace may contain retrieved text. An approval interface may show more context than the reviewer needs. A downstream ticket may retain generated personal data under a different policy.
Minimise before sending
Retrieve and transmit the smallest context required for the task. Filter by tenant, role, purpose and document status before content reaches the model. Prefer identifiers or structured fields when full documents are unnecessary. Redact or transform sensitive fields where the result permits it.
Data-in-place patterns can reduce copying, but they do not remove privacy work. They increase the importance of connection scope, runtime authorization, query logging and safe failure when permissions change.
- Purpose-filter sources before retrieval.
- Apply role and tenant scope before content enters context.
- Limit logs to the evidence needed for operation and investigation.
- Define deletion across primary and derived stores.
- Prevent sensitive output from reaching unapproved destinations.
Treat output as a privacy surface
Generated output can reproduce, infer or combine personal information. Validate output before it is stored or sent, especially where the system can contact people, change records or produce profiles. Separate a model's confidence from the accuracy or lawfulness of the underlying information.
Human approval needs a clear review task and appropriate access. Showing every underlying record to every reviewer is not privacy by design.
Prove configuration and operation
Vendor documentation describes available controls. The enterprise must still evidence which controls are configured for the deployed system: region, retention, training use, sub-processors, access, encryption, logging and deletion. Keep the evidence current as services change.
Monitor policy drift in practice. Users may paste new categories of data, teams may reuse the system for a different purpose, and integrations may broaden access. Privacy review belongs in managed operation, not only procurement.
If the data map ends at the model endpoint, it ends before the system does.
Questions leaders ask
Does an AI model store every prompt?
Behavior varies by provider, product, contract and configuration. Verify the exact service's retention, training, logging and deletion terms rather than assuming.
Is retrieval-augmented generation private?
RAG can support controlled access, but privacy depends on source permissions, retrieval filters, index design, model transmission, logs, output and user access.
What is data minimisation for AI?
Using only the personal data necessary for the defined result and limiting copies, context, retention and access throughout the system.
Primary references
- Regulation (EU) 2016/679 — General Data Protection Regulation — EUR-Lex
- Opinion on AI models and GDPR principles — European Data Protection Board
- Generative AI Profile — NIST AI 600-1 — National Institute of Standards and Technology
Continue reading: AI and GDPR Compliance: A System-by-System Framework.