PII masking in a LiteLLM proxy with the Maskflare guardrail

If you already run a LiteLLM proxy, you have the best place in your stack to mask personal data: every application, every team and every provider goes through it. One guardrail there covers the support bot calling OpenAI, the analytics job calling Claude and the experiment someone started on Groq last week.

This guide sets up the Maskflare guardrail, explains what happens before and after each call, and compares it fairly with LiteLLM's built-in Presidio integration.

How do you add the Maskflare guardrail to LiteLLM?

Install the guardrail extra next to the proxy. LiteLLM needs Python 3.10 or newer:

pip install "maskflare[litellm]" "litellm[proxy]"

Create a Redaction-purpose key in the Maskflare console (Configure, API keys, Create key, Purpose: Redaction). Then add the guardrail to config.yaml. This is a complete, minimal config:

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: maskflare
    litellm_params:
      guardrail: maskflare.integrations.litellm.MaskflareGuardrail
      mode: [pre_call, post_call]
      default_on: true
      api_key: os.environ/MASKFLARE_API_KEY   # a Redaction-purpose key
      fail_policy: closed

Start the proxy and send it a request with something worth masking:

export MASKFLARE_API_KEY=mf_...
export OPENAI_API_KEY=sk-...
export LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)
litellm --config config.yaml

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user",
       "content": "Write two lines to ada@example.com confirming the CHF 2,400 refund."}]}'

OpenAI receives mf_mask_ tokens in place of the email address. Your client gets the answer with ada@example.com back in it. Other options: api_base (defaults to MASKFLARE_BASE_URL, otherwise the public API), timeout, max_retries and max_concurrency.

What happens in pre_call and post_call?

• pre_call: mask. Every message text and tool-call argument string is sent to Maskflare's /v1/mask, up to max_concurrency (default 8) at once, and replaced with its masked form before LiteLLM calls the model.
• post_call: restore. mf_mask_ tokens in the answer's text and tool-call arguments are resolved through /v1/unmask and put back, streamed or not.
• Streams are restored chunk by chunk. A token split across chunks is held back until it's whole: at most 39 characters, and only while the text ends like the start of a token. Chat completion streams are restored; other streamed shapes pass through as sent.
• Only tokens minted for the same call are restored. LiteLLM shares one Maskflare key across all of its users. If someone pastes a token from another conversation into a prompt, it comes back as a token, never as another user's value.
• Non-reversible actions stay as they are. If a rule redacts, partially masks, hashes or applies format-preserving encryption, the model's output keeps that form.

Which values count as sensitive is configured per environment on the Maskflare console's Rules page, not in LiteLLM. The guardrail applies whatever the key's environment has: the 574 built-in detectors, compliance profiles such as GDPR or PCI DSS v4.0, your own regex and dictionaries, and Exact Data Match for your customer records.

Should the guardrail fail open or fail closed?

fail_policy decides what happens when Maskflare can't mask a request or restore an answer, because of a network error, a timeout or an outage:

A match of a block rule refuses the call with a 403 whatever the policy. Errors and log lines name a category (blocked, connection, unavailable, rate_limited, auth, rejected, internal) and the API's error code, never text, values, tokens or keys, so your LiteLLM logs don't become a second copy of the data. If you choose open, alert on those warnings: an unmasked request you don't notice is the worst outcome.

How does this compare with LiteLLM's Presidio guardrail?

LiteLLM ships a Presidio guardrail, and it's a reasonable choice. You deploy Microsoft's Presidio analyzer and anonymizer containers, point LiteLLM at them with PRESIDIO_ANALYZER_API_BASE and PRESIDIO_ANONYMIZER_API_BASE, and add a guardrail block:

guardrails:
  - guardrail_name: "presidio-pii"
    litellm_params:
      guardrail: presidio
      mode: "pre_call"
      presidio_language: "en"
      output_parse_pii: true

Per LiteLLM's documentation as of October 2026, output_parse_pii only puts the original values back in place of placeholders after the call; it doesn't run detection on the output. pii_entities_config sets MASK or BLOCK per entity type, presidio_score_thresholds sets confidence cutoffs, and presidio_filter_scope chooses input, output or both. Here's how the two compare:

Pick Presidio if you want a free, self-hosted library you tune yourself and English names in prose are the main risk. (Maskflare offers on-premises deployment for enterprise, but that's a bigger decision than a container.) Pick the Maskflare guardrail if you need checksum-validated IDs across many countries, secrets and credentials, consistent rules shared with other paths (the desktop agent, browser protection, coding-agent hooks) and audit evidence without running another service. The fairest test is your own traffic: run 200 realistic prompts through both and count what each misses and what each over-masks.

Can LiteLLM point at the Maskflare gateway instead?

Yes, and sometimes that's the better fit. Set a model's api_base to your gateway's provider route (for example https://YOUR_GATEWAY_HOST/v1/openai) and its api_key to a Maskflare gateway key. LiteLLM keeps doing routing, retries and cost tracking; the gateway masks the full request body for that provider, gives embeddings deterministic tokens and masks Batch files line by line. The guardrail is simpler when you want one control across every model in the proxy; the gateway route is better when embeddings and batch traffic go through LiteLLM too.

How do you roll it out across teams?

Put the guardrail in with default_on: true but set the environment's rules to alert first. Every request still goes through the guardrail, nothing is changed, and the Maskflare traffic log shows which rules would have fired. After a few days you know which teams send customer data through the proxy and which data types dominate. Then switch the rules to mask, keep fail_policy: closed, and watch the 503 rate for a day: a spike means the proxy can't reach the API, not that masking is wrong.

Rule changes are the other risk. Maskflare can require maker-checker approval, so a second person approves each rule change before it applies, and the audit log exports as NDJSON for up to 366 days if someone later asks why a value went out unmasked.

What are the limits?

• Each request adds API round trips. Masking and restoring call Maskflare's API, so network latency from your LiteLLM host matters more than the roughly 2 ms masking itself takes. Run the proxy close to the API.
• Restoring covers chat completions. Other streamed response shapes pass through unchanged, tokens included.
• One key per proxy. The guardrail uses one Maskflare key, so every LiteLLM user shares one rule set. Run separate guardrails or proxies if teams need different policies.
• Text only. Images in multimodal messages aren't masked.