Enterprise protection, measured on real work
Developed using 57,349 real business files and 114,570 scanned pages. Measured on 3,396 documents and 6,754 pages held out from model training, multilingual in Latin and non-Latin scripts, labelled under human supervision with legal expertise. Snapshot of 26 October 2026; the full report with its method annex is the PDF.
CPU core pipeline: 3,396 documents in 14.5 minutes. Median latency: 0.36 s per document. Over-masking: 3.8% of masked characters. Benchmark and training documents are multilingual, in Latin and non-Latin scripts, and the benchmark was held out from model training; labelling was conducted under human supervision with legal expertise. OCR and optional semantic processing are measured separately.
One workload. Nine systems.
The same 3,396 real business documents and 6,754 pages, the same extracted text and the same written policy for every system, in one configuration each. Protection: higher is better. Over-masking: lower is better.
| System / publisher | Personal data protected | Over-masking | Measured environment | Elapsed |
|---|---|---|---|---|
| Salus core | 98.3% | 3.8% | Local CPU32-vCPU server / no GPU | 14.5 min |
| GPT-5.6 Sol | 96.58% | 0.9% | Hosted APIOpenAI API / cloud | ≈38 min |
| Claude Sonnet 5.5 | 94.21% | 0.1% | Hosted APIAnthropic Batch API / cloud | ≈16 min |
| Gemma 4 31B31B parameters | 93.33% | 0.7% | Self-hosted GPU1× H100 80 GB / FP8 | 71.9 min |
| Qwen3.8-27B27B parameters | 92.65% | 0.7% | Self-hosted GPU1× H100 80 GB / FP8 | 84.3 min |
| Llama 3.3 70B70B parameters | 84.23% | 2.2% | Self-hosted GPU2× H100 80 GB / FP8 | 38.1 min |
| GLiNER-PII≈0.3B parameters | 58.88% | 17.1% | Local CPUSame 32-vCPU server | 31.5 min |
| Presidio 2.2.365 | 49.66% | 63.8% | Local CPUSame 32-vCPU server | 1.2 min |
| OpenAI Privacy Filter1.5B total / 50M active parameters | 34.18% | 23.7% | Local CPUSame 32-vCPU server | 42.1 min |
About +1.7 percentage points more personal-data protection than the closest comparator, GPT-5.6 Sol. Salus finished ahead of every tested LLM in these run configurations. All four CPU systems used the same 32-vCPU server; the text core needed no GPU. Elapsed times include each run's concurrency and API or batch boundaries and are not a matched-hardware latency test. Release dates identify model launches, not evaluation dates.
Category results
Each score is the share of required occurrences whose letters and digits were fully protected; the predicted type and exact span boundaries need not match. Scores are occurrence-weighted.
| Category | Values | Score | Protected |
|---|---|---|---|
| Person | 32,561 | 0.9886 | 98.86% |
| Organisation | 31,837 | 0.9686 | 96.86% |
| National ID | 6,274 | 0.9914 | 99.14% |
| Date of birth | 5,731 | 0.9895 | 98.95% |
| Address | 2,932 | 0.9533 | 95.33% |
| Passport | 2,487 | 0.9823 | 98.23% |
| Nationality | 2,462 | 0.9533 | 95.33% |
| Place of birth | 2,092 | 0.9924 | 99.24% |
| Bank account | 1,743 | 0.9461 | 94.61% |
| Phone | 1,490 | 0.9779 | 97.79% |
| IBAN | 1,479 | 0.9865 | 98.65% |
| Tax ID | 1,042 | 0.9875 | 98.75% |
| Customer number | 621 | 0.9710 | 97.10% |
| Location | 619 | 0.9806 | 98.06% |
| 410 | 0.9732 | 97.32% | |
| Licence plate | 123 | 0.9756 | 97.56% |
| IP address | 49 | 1.0000 | 100.00% |
| Social security (SGK) | 35 | 1.0000 | 100.00% |
| Driving licence | 14 | 1.0000 | 100.00% |
| Credit card | 13 | 1.0000 | 100.00% |
| UETS | 4 | 1.0000 | 100.00% |
| Website (URL) | 1 | 1.0000 | 100.00% |
| Personal data (excluding organisations) | 62,290 | 0.983 | 98.3% |
| All values (including organisations) | 94,127 | 0.9781 | 97.81% |
The benchmark contains 94,127 required values; excluding 31,837 organisation occurrences leaves 62,290 personal-data values. One catch-all category, Other identifiers (108 values, score 0.4352), is included in both aggregates but not shown as a row; it holds personal identifiers that fit no named category and is a rule-coverage item for the next release. Small categories describe this sample and do not establish universal format coverage. Customer-number formats can be adjusted on request through deployment-specific rules.
How the benchmark was built and measured
A summary of the two-page method annex in the report.
Source archive and families
57,349 real business files, including legal, financial and administrative records, multilingual in Latin and non-Latin scripts; the 114,570 scanned pages were transcribed once by the Salus vision-language reader. Every file belongs to a family by case code or source folder, about 2,000 families; training, validation and test sets are split by family hash, never by file, and the benchmark's families are excluded from training.
Leakage control and one extraction
Pages whose normalised text equals a benchmark page are removed from training, and every training export is checked for near-duplicate pages; a failing export is not used. Office files are read from their native text and scans by the vision-language reader; the identical files are given to every compared system.
Written policy, two passes, adjudication
22 required categories plus neutral classes, with explicit rulings; the policy was written with legal expertise. Each page is labelled in two independent passes, blind to each other, under human supervision with legal expertise; agreement on required spans before adjudication was F1 0.948 on the core set and 0.929 on the held-out additions. Every disagreement is adjudicated against the page image; more than 6,000 were resolved this way.
Scoring
A required value counts as protected only when none of its letters or digits remain visible. Over-masking is unnecessary masking as a share of all masked characters, one definition for all nine systems; its complement, 96.2% for Salus, is masking precision at character level, not entity-type precision.
Comparator protocol
Language models received the text in fixed character windows with the written policy and returned the values to mask as structured JSON; hosted models through their published APIs, open-weight models at temperature 0 in FP8 on H100 hardware, specialist CPU tools on the same 32-vCPU server in their published default configurations. Versions and release dates were verified against primary sources.
Training regime and verification
Salus-specific models are trained only on real business documents from the training families, labelled under the same policy. Each model has one job; inference is deterministic: fixed weights, pinned thresholds, no sampling. Candidate checkpoints are scored on this benchmark before any replaces the release. Every figure carries its snapshot date; portable verifiers recompute the arithmetic and record SHA-256 hashes of the source files. Model families, parameter counts and serving software are not disclosed.
The full report
Seven pages: the comparison, all category results, workload statistics, and the two-page method annex on corpus construction, labelling, scoring, comparator protocol, training regime and verification. The PDF arrives by email; the link is valid for seven days. Snapshot of 26 October 2026.