What Is Reversible Tokenization?

How does reversible tokenization work?

Say a bank's support app sends this to a language model: "Ada Lovelace (ada@example.com) disputes a CHF 2,400 charge on card 4532 7781 0021 9047." With tokenization in the path, five things happen in a few milliseconds:

1. Detect. The name, the email and the card number (which passes the Luhn check) are found.
2. Tokenize. Each value gets a token, for example mf_mask_170b... for the name.
3. Store. The mapping from token to original value is written, encrypted, to a token store (often called a vault).
4. Forward. The model receives "mf_mask_170b... (mf_mask_4e21...) disputes a CHF 2,400 charge on card mf_mask_d7e0...".
5. Restore. When the answer comes back and repeats a token, it's swapped for the original before your user reads it.

Your app sends:   Ada Lovelace (ada@example.com) disputes a CHF 2,400 charge
Provider gets:    mf_mask_170b... (mf_mask_4e21...) disputes a CHF 2,400 charge
Provider answers: Suggested reply to mf_mask_170b...: we've opened a dispute...
Your user reads:  Suggested reply to Ada Lovelace: we've opened a dispute...

The amount stays in clear text on purpose. It isn't identifying on its own, and the model needs it to write a sensible reply. Choosing what to leave alone is as much a part of tokenization policy as choosing what to replace.

How is tokenization different from encryption and hashing?

The key difference from encryption: a random token has no mathematical relationship to the value. Steal a million tokens and you have nothing. The risk moves to the token store, which is why it's encrypted and access-controlled. For a broader comparison, see PII redaction vs data masking vs tokenization.

Random or deterministic tokens?

Random tokens give each occurrence its own token, which reveals the least. Deterministic tokens give the same value the same token every time. That's what you need for embeddings: if "Ada Lovelace" became a different token in every document, a vector search for her would never match. The trade-off is linkability. Anyone who sees many deterministic tokens can tell which records belong to the same person, even without knowing who that is.

Maskflare uses deterministic tokens for embeddings requests, and those requests fail closed: if masking can't run, the request isn't sent. In the Python SDK you can ask for deterministic masking with deterministic=True. More in RAG and embeddings.

Where does PII tokenization help most?

• AI prompts and responses, where the model needs structure but not identities, and the reader needs the real values back.
• Card data. Tokenizing card numbers is a long-standing way to keep them out of systems that don't need them. See PCI DSS.
• Analytics and data sharing, where analysts count and join records without seeing names.
• Customer support tooling, where agents work tickets that third-party tools also process. See customer support AI.

Is tokenized data still personal data?

Under GDPR, usually yes for you. Reversible tokenization fits the definition of pseudonymization in Article 4(5): the data can't be attributed to a person without additional information that's kept separately and protected. Pseudonymized data is still personal data for whoever can reverse it. Whether it's personal data for a recipient that can't reverse it depends on the facts; see pseudonymization vs anonymization. Not legal advice.

How does Maskflare tokenize PII?

• Mask is the usual action for personal data. Each rule can instead redact, keep part of the value, hash, encrypt with FF1, alert or block.
• Restoring is switched on per gateway key and works while streaming, event by event, and in the browser for ChatGPT, Claude, Gemini, DeepSeek and Perplexity.
• Originals are kept encrypted for 30 days for restoring. Prompts and responses themselves aren't logged or stored.
• The SDKs expose the same operations directly: mask, unmask and unmask_text in Python (pip install maskflare), and mask, unmask and unmaskText in TypeScript (npm install @maskflare/sdk).

There's a step-by-step walkthrough in restore PII in LLM responses.

When is tokenization the wrong choice?

When nobody will ever need the value back. Application logs, analytics exports and documents for publication are better served by plain redaction or a keyed hash: there's no store to protect and nothing to restore. Tokenization is also overkill for values the model genuinely needs to read, such as an amount or a product name. And for credentials it's the wrong tool entirely. A live API key shouldn't be swapped for a token and sent on; the request should be blocked and the key rotated.

What are the limits?

• Tokens expire. Maskflare tokens resolve for 30 days. An answer stored with tokens and opened after that won't restore.
• Models can mangle tokens. A model may occasionally reformat, shorten or drop a token, and a mangled token can't be restored.
• Context still leaks. Tokenizing the name doesn't hide "the only female board member in the Zurich office".
• The store is a target. Whoever holds the token store holds the originals, so its access controls matter as much as the tokenization itself.