TL;DR: The safest way to use large language models (LLMs) with real business data is to minimize what the model ever sees. Redact sensitive tokens at ingress with context-aware detection, preserve meaning with semantic placeholders, and restore originals only where policy allows—leaving auditable evidence the whole way. This pattern preserves utility and privacy, and it scales.
Why AI privacy changed in 2025
Traditional privacy programs focused on systems of record: CRMs, EMRs, data warehouses. LLMs flipped the script. Now the riskiest surfaces are ad hoc prompts, intermediate chains, tool calls, and long-lived outputs that bounce between tickets, chats, analytics, and wikis. The same employee who would never upload a spreadsheet to a public forum may paste its contents into a prompt without a second thought. Multiply that by hundreds of workflows, and you have a new risk topology.
Even when vendors offer enterprise controls and retention options, the safest assumption is that anything you send could be kept longer than intended, backed up, or discoverable in litigation. That’s why organizations increasingly treat pre-model minimization as a non-negotiable control, not an optional add-on.
The core idea: minimize at ingress, restore on egress
Redaction is the practice of removing or replacing sensitive information before it reaches an untrusted boundary. For LLMs, the boundary is the model context window. Instead of sending raw names, account numbers, claim IDs, or API keys, you replace them with semantic placeholders—tokens that carry structure and meaning, but reveal nothing sensitive.
Examples:
"Contact Sarah Chen at 555-0147"
→"Contact <PERSON#A> at <PHONE#1>"
"Card 4111-1111-1111-1111 expires 06/27"
→"Card <PAN#1> expires <DATE#B>"
"API_KEY=sk-live-abc..."
→"API_KEY=<SECRET#API>"
(and drop restoration entirely for secrets)
After inference, a restoration map—stored in a separate, access-controlled vault—can re-insert originals into approved destinations (e.g., a signed letter), while outputs destined for knowledge bases stay redacted by default.
What makes redaction "smart"?
Regex alone can’t handle the ambiguity and variety of natural language. Smart redaction blends high-precision patterns with context models to detect 50+ entity types: PII (names, emails, phones, addresses), PHI (MRN, diagnosis, treatment), financial (PAN, IBAN, account/routing), employment/education IDs, and technical identifiers (device IDs, IPs, tokens). It also respects domain context: a nine-digit number in a US payroll email is likelier to be a SSN than in a product manual; a six-digit string near "OTP" is probably a one-time code, not an invoice number.
Placeholders that preserve utility
Great placeholders have three properties:
- Semantic: They encode entity type and relationships (e.g.,
<PERSON#A>
and<PERSON#B>
maintain who’s who across a document). - Stable in scope: Deterministic inside a session so the model can track references, but not reusable across sessions without authorization.
- Human-readable: Auditors should understand a redacted record without decoding.
Architecting the redaction pipeline
A production-grade setup usually includes:
- Gateway: A network or SDK layer that intercepts all model traffic. It performs redaction, logging, policy enforcement, and request shaping (timeouts, retries, rate limits). Direct calls to vendor APIs are blocked from production networks.
- Policy engine: Declarative rules assigning actions (mask, drop, allow, hash) per entity type, tenant, and environment (dev vs. prod). Changes require review and leave an immutable trail.
- Restoration service: A separate microservice with stricter access controls and key management. It uses short-lived, least-privilege tokens and emits restoration events for audit.
- Logs & metrics: Structured events that count detections, policy decisions, latency, and errors—not raw prompts. Dashboards show false positive/negative rates by entity and by team.
Governance that scales
Paper policies alone won’t stop pasting. Treat your redaction gateway as the paved road: faster, easier, and safer than going around.
- Golden paths: Official SDKs and CLI tools everyone uses because they improve reliability and developer experience.
- Guarded exceptions: If a team truly needs raw data for an experiment, time-box it in an isolated sandbox with auto-expiry and explicit sign-off.
- Change control: Policy updates follow the same rigor as code—pull requests, reviews, tests with seeded datasets, and staged rollout.
Metrics that prove it works
Measure both privacy and business outcomes:
- Detection quality: Precision/recall per entity, with labeled test sets and drift monitoring.
- Restoration accuracy: End-to-end success rate and time to restore, broken down by workflow.
- Leak rate: Incidents per 10k AI requests; mean time to detect and contain.
- Adoption: Percentage of AI calls via the gateway; number of exceptions open.
Common pitfalls (and fixes)
- Over-masking kills meaning: Tune thresholds; use allowlists (e.g., public brand names) to reduce false positives.
- Logs quietly re-introduce risk: Ban raw bodies; add CI checks that fail builds when risky logging calls appear.
- Secrets treated like PII: Never restore secrets. Treat detection as an incident: block, rotate, alert.
- One-region thinking: Add data residency controls; run redaction locally and keep restoration keys in-region.
Case studies (anonymized)
Healthcare note summarization: A provider redacts MRN, names, and contact info before model calls; coding suggestions and summaries remain useful because placeholders preserve roles and chronology. Restoration inserts member IDs only in communications destined for payers.
Fintech support automation: A bank masks PAN, account numbers, and balance figures in prompts; the model drafts empathetic replies with placeholders; a post-processor restores only non-sensitive values. Logged events show zero PAN exposure outside PCI-scoped systems.
Why this future is durable
Vendor features, regulations, and case law will evolve. The redaction-first architecture is resilient to that volatility because it relies on your boundary controls. When requirements tighten, you update policy; when they loosen, you keep the safety margin.
Implementation checklist
- Map prompts/outputs across your top 10 AI workflows.
- Define entity catalog and policy actions.
- Deploy a gateway and make it the only path to models.
- Add restoration service and separate key management.
- Instrument metrics and alerts; seed test data; tune.
- Roll out with training, paved SDKs, and guarded exceptions.
The bottom line
AI privacy in 2025 is less about where data rests and more about what flows through models, chains, and tools. Redaction with reversible placeholders gives you the best of both worlds: useful AI and controllable risk. Build it once, and every workflow gets safer by default.
Questions about AI security?
Our experts are here to help you implement secure AI solutions for your organization.
Contact Our Experts