Protegrity-PII outperforms Claude Fable 5, OpenAI Privacy Filter, AWS Comprehend and Microsoft Presidio on both recall and precision in the new PII-SCOPE benchmark, detecting 69 types of personal information
MENLO PARK, Calif. – Protegrity, a global leader in AI and data security, today announced the availability of Protegrity-PII, a new open-weights model for detecting and classifying personally identifiable information (PII) in free-form text. The compact model, just 278M parameters, is now available on Hugging Face, giving AI engineering and security teams access to run and evaluate Protegrity’s approach to PII detection in their own environments.
Released with the model are PII-SCOPE, a public benchmark for fine-grained PII detection, and the Protegrity-PII Technical Report on arXiv. Together, the model, benchmark and research provide access to the technology, test data and methodology behind Protegrity-PII.
“You don’t get ahead in AI security by putting an AI label on yesterday’s technology. You get there with deep security competence, exceptional AI talent and the willingness to build something genuinely new,” said Michael Howard, CEO, Protegrity. “That’s why I’m so excited about Protegrity-PII. We’re opening the model, the benchmark and the research so people can see the results for themselves. This is another example of Protegrity working at the bleeding edge of AI and data security, and I couldn’t be more excited about where it takes us next.”
Protegrity-PII is a token-level, encoder-only classifier built on XLM-RoBERTa. It recognizes 69 PII entity types, extending beyond names and contact information to categories including account and card data, device and tracking identifiers, health information, credentials, immigration status, and biometric and neural data. Because the model classifies tokens rather than following prompts, text presented for inspection is treated as content to label, not instructions to execute, so the model isn’t vulnerable to prompt injection attacks the way instruction-following models are.
“PII detection sounds like a solved problem until you look closely at how existing approaches perform against the breadth and complexity of sensitive data found in real-world environments,” said Hardeep Bassi, Lead Machine Learning Engineer, Protegrity, and lead author of the research. “By releasing Protegrity-PII, PII-SCOPE and the supporting research, we are giving AI and security engineers the opportunity to run the model, test it against an open benchmark and evaluate the results for themselves. Our goal is to advance PII detection with broader coverage, higher recall and greater efficiency to protect sensitive data within AI workflows.”
A New Open Model for PII Detection in AI Workflows
As AI workflows and agents interact with more enterprise data, PII can move across prompts to hosted models, tool calls and APIs, agent responses, traces and logs, vector stores, and training and evaluation datasets. Protegrity-PII is designed to run inline on an organization’s own infrastructure, enabling PII detection at multiple points across those workflows.
-
Open-weights availability. Protegrity-PII is available on Hugging Face.
-
Broader entity coverage. The model is trained directly on 69 PII entity types, more than AWS Comprehend (25), Microsoft Presidio (17) and OpenAI Privacy Filter (8).
-
Recall-focused results. Achieved 0.915 910 typed recall at 0.907 902 precision on PII-SCOPE, higher than AWS Comprehend with 0.465, OpenAI Privacy Filter with 0.387 and Microsoft Presidio with 0.386.
-
Efficient self-hosted processing. Processed 1M characters in 0.49s at an estimated cost of $0.0005 using batched streams on a single RTX 6000 Blackwell GPU.
-
Production relevance. The open-weights model is a smaller version of the PII detector used in Protegrity’s commercial products.
An Open Benchmark and Supporting Technical Report
PII-SCOPE contains 8,030 records across healthcare, retail, government, finance, insurance, HR, legal and technology. Its formats include emails, letters, medical records, resumes, chat logs, receipts, invoices and support tickets. The benchmark averages 14.31 PII entities and 8.17 distinct entity types per row, creating a demanding test for models that must identify multiple forms of sensitive information within the same record.
Protegrity evaluated Protegrity-PII against OpenAI Privacy Filter, GLiNER2.5 Multi, AWS Comprehend, Microsoft Presidio and Rampart, with Claude Fable 5 included separately as a frontier-model reference. Protegrity-PII recorded the highest typed recall among the non-LLM detectors across all five datasets evaluated. It exceeded the frontier reference on PII-SCOPE and SPY and was on par with it on AI4Privacy, Nemotron-PII and Gretel.
The Protegrity-PII Technical Report is authored by Hardeep Bassi, Hui Ma, Luís Seabra, Koowum Joshi, Daniel Pereira and Averell Gatton. The report describes the model architecture, benchmark methodology, entity taxonomy, evaluation results, processing speed and estimated cost.
Access Protegrity-PII
Explore Protegrity on Hugging Face for the Protegrity-PII model and related resources.
Read the Protegrity-PII Technical Report on arXiv for the complete methodology and benchmark results.
About Protegrity
At Protegrity, we provide data protection for AI and modern applications, securing sensitive data while enabling enterprises to unlock insights, accelerate innovation, and meet global compliance with confidence. From emerging AI to quantum threats, our flexible and forward-looking approach ensures data protection evolves as technology advances. With developer-friendly tools such as SDKs, APIs, and containerized services, we make it easy for teams to embed strong data protection into applications and workflows, securing innovation at scale.
Media Contact
Touchdown PR on behalf of Protegrity
[email protected]