Privacy and PII handling

The big idea

A support team adds a feature that drafts replies to customer tickets. Each ticket holds a name, an email address, sometimes a phone number and occasionally a full card number pasted by a worried customer. The feature works well. Then someone asks a simple question: where does that card number end up? In the prompt sent to the model provider, in the provider's logs for some days, in your own request logs for 90 days, in the trace your observability tool keeps, and in the vector store that powers "similar tickets". One paste, five copies.

PIIPersonally identifiable information: any data that identifies a person directly (name, email, phone, card or ID number) or can identify them when combined with other data. handling for AI features is mostly about those copies. The model itself is rarely the problem. The pipeline around it multiplies data, and every copy has to be secured, kept only as long as needed, and deleted when a person asks.

Think of handing documents to an outside translator. You would black out account numbers that the translation does not need, ask how long they keep copies, and maybe, for the most sensitive files, hire a translator who works inside your building. The same three questions apply to a model: what do I send, how long is it kept, and who runs it.

How it works

Minimize first

The cheapest protection is not sending the data at all. Ask what the task needs. Drafting a reply to a double charge needs to know there was a double charge, not the card number. Classifying a ticket needs the text, not the customer's address. European data protection law calls this data minimisationThe principle that personal data should be adequate, relevant and limited to what is necessary for the purpose it is processed for (GDPR Article 5)., and it is good engineering anywhere: less data in the prompt means less to leak, log and delete.

Redact, mask or pseudonymize

When personal data is in free text, a detection step finds it and replaces it:

• Masking replaces each value with a type label such as [EMAIL]. Simple, irreversible, and fine when the reply does not need the value.
• PseudonymizationReplacing each personal value with a consistent placeholder, such as EMAIL_1, while keeping a private mapping so the real value can be restored later. replaces each value with a numbered placeholder and keeps a mapping on your side. The model can still say "we will email <EMAIL_1>", and your code fills in the address afterwards. Under GDPR, pseudonymized data still counts as personal data for you, because you hold the key, but it greatly reduces what any outside party sees.

Detection mixes methods. Regular expressions and checksums find well-formatted values: emails, card numbers, phone numbers, ID numbers. Names, addresses and free-form details ("my daughter's school in Leeds") need a named-entity model, and even then some slip through. Open-source tools such as Presidio combine both and say plainly that automated detection gives no guarantee of finding everything.

Retention: how long each copy lives

Every store needs a retention period, chosen on purpose:

• The model provider. Read the provider's data terms. As of 2026, both Anthropic and OpenAI document that standard API inputs and outputs are deleted or kept for abuse monitoring for up to 30 days and are not used for training by default, with exceptions such as content flagged for policy violations. Both offer zero data retention arrangements by agreement, which some features do not support.
• Your own logs and traces. Often the longest-lived copy and the one teams forget. Log placeholder text, not raw tickets, and set an expiry.
• Memory, caches and vector stores. Anything indexed for retrieval or kept as conversation memory is another database of personal data. It needs the same deletion path.

GDPR also gives people a right to have their data erased without undue delay in many cases. If you cannot find every copy of a customer's data, you cannot honour that request.

On-prem or API

A self-hosted modelA model you run on your own servers or in your own cloud account, so prompts and outputs never go to an outside model provider. keeps prompts inside your own infrastructure, which can settle questions about data residency, contracts and third-party processors. The cost is that you now run GPUs, patch the serving stack and usually use a smaller open-weights model than the frontier ones behind an API. A middle ground is a hosted API with a zero-retention agreement, or the same models through your cloud provider in a region you choose. Pick by your obligations, not by habit: a contract, a regulator or a customer's security review often decides it.

Try it

1. At the starting settings, two places hold raw personal data. Which one keeps it longest, and is it the provider or you?
2. Switch to "pseudonymized" and set the logs to store what was sent. How many places now hold raw personal data, and why is the mapping one of them?
3. Turn on "keep conversations for memory or search". What new obligation did that create, even with the self-hosted model?

The pseudonymize-and-restore round trip in code:

# Pseudonymize a support ticket before it goes to a hosted model, then put the
# real values back into the reply on your side. The mapping never leaves you.
# Regular expressions catch well-formatted data; names need an entity model.
import re

PATTERNS = {
    "EMAIL": r"[\w.+-]+@[\w-]+\.[\w.]+",
    "CARD":  r"\b\d(?:[ -]?\d){12,15}\b",
    "PHONE": r"\+?\d[\d -]{8,}\d",
}

def pseudonymize(text):
    mapping = {}
    for kind, pattern in PATTERNS.items():
        for i, value in enumerate(dict.fromkeys(re.findall(pattern, text)), 1):
            token = f"<{kind}_{i}>"
            mapping[token] = value
            text = text.replace(value, token)
    return text, mapping

def restore(text, mapping):
    for token, value in mapping.items():
        text = text.replace(token, value)
    return text

ticket = ("Hi, I'm Dana Reyes. My card 4111 1111 1111 1111 was charged twice. "
          "Email me at dana.r@example.com or call +44 20 7946 0958.")
safe, mapping = pseudonymize(ticket)
print("sent to the model:\n ", safe)

reply = "Sorry about that! We will refund the duplicate charge and email <EMAIL_1>."
print("reply shown to the agent:\n ", restore(reply, mapping))
print("kept locally:", sorted(mapping))
print("still in the sent text:", [w for w in ["Dana", "Reyes"] if w in safe])

sent to the model:
  Hi, I'm Dana Reyes. My card <CARD_1> was charged twice. Email me at <EMAIL_1> or call <PHONE_1>.
reply shown to the agent:
  Sorry about that! We will refund the duplicate charge and email dana.r@example.com.
kept locally: ['<CARD_1>', '<EMAIL_1>', '<PHONE_1>']
still in the sent text: ['Dana', 'Reyes']

The email, card and phone number never leave your side, and the reply still reads naturally once restored. The last line is the warning: the customer's name went through untouched, because no regular expression can recognize names. Add an entity model for names and addresses, and treat redaction as reducing risk, not removing it.

Going deeper

Models can memorize

Data that reaches a model's training set can come back out. Researchers extracted hundreds of verbatim training sequences from a public language model, including names, phone numbers and email addresses, some of which appeared in only one training document. That is one reason providers' default API terms exclude training on customer data, and a strong reason not to fine-tune your own model on raw tickets. Redact or pseudonymize training data the same way you do prompts.

Personal data leaks through the side doors

• Retrieval. A vector store built from all tickets can return one customer's details in another customer's answer. Filter retrieval by the current user's permissions, not just by similarity.
• Tools and agents. An agent with database access can fetch more than the task needs. Scope its queries to the current customer.
• Prompt injection. An attacker can try to make the model reveal personal data from its context ( Prompt injection and data exfiltration ). Less data in the context means less to steal.
• Outputs. Run an output check for personal data before a reply is shown or stored ( Guardrails ).

Write it down

Keep a short data map for each AI feature: what personal data flows in, which stores hold it, for how long, which processors see it and how deletion works. Privacy reviews, security questionnaires and deletion requests all start from that list, and building it usually reveals a copy nobody knew about.

Trade-offs

Common mistakes

• Redacting the prompt but logging the raw ticket, so the longest-lived copy is the unprotected one.
• Assuming regular expressions catch everything. Names, addresses and free-text details need entity detection, and some will still slip through.
• Building a retrieval index or memory store from customer data with no deletion path.
• Fine-tuning on raw support transcripts, then discovering the model can repeat them.
• Never reading the provider's data terms, or not checking which features fall outside a zero-retention agreement.

In the wild

Check yourself

Recap