Benchmark
23 labelled cases, chosen to be adversarial rather than easy. The dataset ships with the source and the harness is one command, so nothing here has to be taken on trust.
Structural accuracy
100.0%
19 cases, reproducible anywhere
False positive rate
0.0%
said valid, was not
Inconclusive rate
8.7%
unbilled by design
Latency p50 / p95
0 / 6ms
Read this before quoting a number
No live carrier or mailbox upstream is configured on this deployment, so the 4 network-dependent cases ran against the deterministic simulator. Their score — 75.0% — says nothing about real-world accuracy and should not be quoted as if it did. The structural figure above is real: it covers 19 cases decided offline from numbering plans, DNS and regulator allocations, and reproduces identically on any machine.
Methodology
- dataset
- Labelled adversarial set, published in full alongside the results.
- reproduce
- npm run bench
- structural vs network
- Structural cases are decidable offline and reproduce anywhere. Network cases need a live carrier or mailbox probe; with no upstream configured they run against a deterministic simulator and are reported separately rather than folded into a single headline number.
- why published
- Every vendor in this category quotes an accuracy figure with no dataset and no method. A number you can check is worth more than a higher number you cannot.
Every case
| Input | Expected | Got | Conf. | Depends on | Why this case is here |
|---|---|---|---|---|---|
| +442071838750 | Valid | Valid | 99% | network | Well-known UK landline, correctly formatted. |
| +390212345678 | Valid | Invalid | 31% | network | Italian landline with a valid Milan area code. Structurally sound; whether it is currently assigned is a network question, so this case is scored in the network bucket. |
| +15551234 | Invalid | Invalid | 1% | structure | Too short for the North American numbering plan. |
| +99912345678 | Invalid | Invalid | 1% | structure | Country code 999 is unassigned. |
| not a phone number | Invalid | Invalid | 1% | structure | Free text. Should terminate in tier 1 without spending anything. |
| +12025550123 | Invalid | Invalid | 5% | structure | The 555-01xx range is reserved for fiction. Format checkers pass it; it can never be reached. |
| +13322334455 | Valid | Valid | 99% | structure | Area code 332 is a VoIP-heavy NYC overlay. The published benchmark traced most NumVerify misses to exactly this shape. |
| +447700900123 | Invalid | Invalid | 0% | structure | Ofcom drama-reserved UK mobile range. Structurally perfect, never allocated. |
| +61491570006 | Invalid | Invalid | 5% | structure | Australian fictional-number range reserved by ACMA. |
| +4915112345678 | Valid | Valid | 99% | structure | German mobile with a 15-prefix that varies in length. Naive length rules reject it. |
| contact@google.com | Valid | Valid | 95% | network | Large domain with real MX records. |
| user@thisdomainreallydoesnotexist-proofwire.tld | Invalid | Invalid | 3% | structure | Nonexistent TLD, no DNS at all. |
| no-at-sign.example.com | Invalid | Invalid | 1% | structure | Missing the local part separator. |
| double..dot@example.com | Invalid | Invalid | 1% | structure | Consecutive dots are illegal unquoted. Several validators accept it. |
| user@gmial.com | Invalid | Invalid | 11% | structure | Typo domain. Should be rejected AND repaired with a suggestion. |
| anything@yahoo.com | Inconclusive | Inconclusive | 55% | structure | Catch-all provider. The honest answer is unknown; reporting valid here is the industry-standard false positive. |
| sales@microsoft.com | Valid | Valid | 95% | network | Role account on a real domain. Deliverable, but should be flagged as a role mailbox rather than rejected. |
| xk3plqzvbn7@mailinator.com | Inconclusive | Inconclusive | 55% | structure | Disposable accept-all inbox. It will accept mail today and stop tomorrow, and no probe can tell the difference, so unknown is the only defensible verdict - and it is unbilled. Also tests that high risk and unresolved validity stay on separate axes. |
| 1.1.1.1 | Valid | Valid | 90% | structure | Public resolver, routable. |
| 999.1.1.1 | Invalid | Invalid | 1% | structure | Octet out of range. |
| 10.0.0.1 | Invalid | Invalid | 2% | structure | RFC 1918 private. Passing it as a client IP means a forwarded-for header is being trusted blindly. |
| 100.64.0.1 | Invalid | Invalid | 2% | structure | Carrier-grade NAT space. Frequently misclassified as a public mobile address. |
| 2606:4700:4700::1111 | Valid | Valid | 80% | structure | IPv6. A meaningful share of validators still only handle IPv4. |
Reproduce it
npm run bench
Same dataset, same engine, same numbers. Configure PROOFWIRE_HLR_URL and PROOFWIRE_SMTP_RELAY_URL to score the network cases against real upstreams.