419 Automated Tests: Production PII Detection Verification
Research Source
PII anonymization vendors claim high accuracy but rarely publish test results. Customers cannot verify detection quality before purchasing. There is no industry-standard benchmark for PII detection accuracy. The result: organizations deploy PII tools without knowing their actual detection rate, discovering failures only when PII leaks through.
Executive Summary
PII vendors claim high accuracy but publish no test results . Organizations deploy tools without knowing actual detection rates. Failures are discovered when PII leaks — not during evaluation.
anonym.legal publishes a 419-test suite with 100% pass rate, covering 13 milestones, 48 languages, 4 browsers, and 35 security tests. Full test results are publicly available at /docs/testing/pii-detection.
The Problem: Unverified Accuracy is Unverified Compliance
GDPR Article 32 requires 'appropriate technical measures' for data protection. If an organization deploys a PII detection tool claiming 95% accuracy but actual accuracy is 70%, the organization has a 30% compliance gap it doesn't know about. Without published test results, every accuracy claim is marketing — not engineering. Organizations need verifiable, reproducible test results to assess whether a PII tool meets their compliance requirements.
Irreducible truth: An accuracy claim without published test results is not a technical specification — it is marketing copy. Verifiable accuracy requires published tests with reproducible methodology, covering all claimed entity types and languages.
The Solution: How anonym.legal Addresses This
13 Test Milestones
The test suite covers: M01 Basic PII detection, M02 Entity filtering, M03 Multi-language (48 languages), M04 Batch processing, M05 File formats, M06 Custom entities, M07 Encryption/decryption, M08 Office Add-in, M09 API endpoints, M10 MCP Server, M11 Chrome Extension, M12 Desktop integration, M13 Security tests.
48 Language Coverage
Each of the 48 supported languages is tested with language-specific PII examples. German Personalausweis numbers, Japanese My Numbers, Arabic names, Hebrew addresses, Korean RRNs — all verified with real-world format examples.
35 Security Tests
SSRF protection, ZK auth verification, timing-safe comparisons, CSRF protection, rate limiting, Retry-After headers, API key validation, session management, and more. Security tests verify that PII processing cannot be exploited.
Public Dashboard
Full test results published at
/docs/testing/pii-detection
with 13 milestone reports, 151 screenshots, and token usage tracking. Anyone can verify the 419/419 (100%) pass rate.
Compliance Mapping
This feature directly supports GDPR Article 32 (security of processing — documented technical measures), ISO 27001 Annex A.14 (system testing), and procurement requirements for evidence-based vendor evaluation.
anonym.legal's GDPR, HIPAA, PCI-DSS, ISO 27001 compliance coverage, combined with Hetzner Germany, ISO 27001 hosting, provides documented technical measures organizations can reference in their compliance documentation.
Product Specifications
| Specification | Value |
|---|---|
| Entity Types | 320+ |
| Detection | 3-layer hybrid: Presidio + NLP + Stance classification |
| Test Coverage | 100% (419/419 tests) |
| Languages | 48 |
| Anonymization Methods | Replace, Redact, Mask, Hash (SHA-256/512), Encrypt (AES-256-GCM) |
| Platforms | Web App, Desktop, Office Add-in, Chrome Extension, MCP Server, REST API |
| Pricing | Free €0, Basic €3, Pro €15, Business €29 |
| Hosting | Hetzner Germany, ISO 27001 |
| Compliance | GDPR, HIPAA, PCI-DSS, ISO 27001 |
Limitations & Considerations
Integration Complexity: Organizations implementing this solution should expect comprehensive organizational assessment, compliance framework evaluation, and technical infrastructure review before deployment. Integration complexity varies based on existing systems, data workflows, and regulatory requirements.
Data Volume Scaling: Performance characteristics vary with data volume, document format diversity, and entity pattern complexity. Organizations processing high-volume document streams should conduct benchmark testing with representative samples to validate throughput and accuracy targets.
Team Training Requirements: Requires 2-4 weeks of onboarding for security and compliance teams to configure custom entity patterns, establish organizational policies, and integrate with existing workflows. Dedicated privacy engineering resources accelerate deployment.
Not for: Organizations without dedicated privacy engineering resources or regulatory compliance mandates may find simpler solutions more cost-effective. Best suited for teams with stringent data protection requirements (GDPR, HIPAA, CCPA).