Guide
Structured and unstructured PII need different detection methods
- 01
Identify sensitive values
- 02
Replace with stable tokens
- 03
Restore only inside your boundary
Direct answer
Structured PII detection uses known fields, schemas, and validation rules to identify sensitive values. Unstructured PII detection must infer meaning from surrounding language and document context. Enterprise systems typically need deterministic rules for structured data and contextual detection for free text.
Structured data provides useful boundaries
A database column named email_address or a JSON field named customerId already provides evidence about the value it contains. Schema-aware policy can classify the field, validate its format, and apply a predictable transformation.
Structure does not guarantee accuracy. Values can appear in the wrong field, nested payloads can change between versions, and free-text fields often sit beside well-defined columns. Detection should combine schema knowledge with validation rather than trust field names alone.
Unstructured text requires context
Names, addresses, and account references may not have a unique format. Context determines whether a number is a customer identifier, a date, an invoice, or an ordinary quantity. Pattern rules remain useful for strongly formatted data, while contextual models help with ambiguous entities.
A combined pipeline can use field policy, exact dictionaries, pattern and checksum validation, named-entity recognition, and organization-specific rules. Each detector should return enough context and confidence for the policy layer to choose an action.
- Schema and field classification
- Patterns, checksums, and format validation
- Contextual entity recognition
- Organization-specific dictionaries and identifiers
- Confidence thresholds and policy exceptions
Evaluate by data type and context
An overall accuracy percentage can hide important failures. Teams should measure precision and recall separately for each data type, language, source, and transformation. A missed national identifier carries a different consequence from an unnecessary name redaction.
MaskFlare's detection pipeline and supported formats are in development. Evaluation should be based on the customer's representative, approved test corpus.
Related PII guidance
Frequently asked questions
Are regular expressions enough for PII detection?
They work well for strongly formatted values but are weaker for ambiguous entities such as names and addresses. Contextual detection and schema information improve coverage.
What is precision in PII detection?
Precision is the share of values flagged as PII that were actually PII. Low precision creates false positives and unnecessary redaction.
What is recall in PII detection?
Recall is the share of actual PII values the system detected. Low recall means sensitive values are missed.