Two missing details
Model scanners raise two kinds of finding. An alarm says "this is dangerous". A warning is a lower-severity or advisory finding that asks a person to look. A warning on a benign file is not an accusation of malware.
Most published false-positive figures leave out two details: whether warnings were counted, and how many files the scanner actually gave a verdict on. Both change the answer. Some tools fail your build on a warning by default, and a "zero false positives" claim can quietly count alarms only. Ours did.
Disclosure: we build Hayward, one of the five scanners below, and we wrote the benchmark harness. Read the table with that in mind.
What we measured
Five open-source scanners on the same 222 benign files: 210 real models from Hugging Face, pinned by hash across 14 formats, plus 12 benign test files the harness generates. A real model counts as benign only if none of Hugging Face's scanners marked it unsafe, no import in it was rated dangerous, and it comes from a known publisher or a widely downloaded repository. That makes them very likely harmless, not proven harmless.
| Scanner | Alarm | Alarm or warning | Verdicts / 222 |
|---|---|---|---|
| Hayward 1.16.1 | 0 of 222 (0%) | 7 of 222 (3%) | 222 |
| picklescan 1.0.5 | 1 of 192 (0.5%) | 38 of 192 (20%) | 192 |
| ModelAudit 0.2.54 | 14 of 222 (6%) | 90 of 222 (41%) | 222 |
| ModelScan 0.8.8 | 0 of 80 (0%) | 0 of 80 (0%) | 80 |
| fickling 0.1.13 | 90 of 94 (96%) | 90 of 94 (96%) | 94 |
Each percentage is over the files that tool gave a verdict on, shown in the last column. Files a tool could not read are counted separately, never as clean. A file counts once, however many findings it gets. Because each tool is scored on a different subset, this compares base installs on one corpus; it is not a like-for-like ranking.
The last column matters as much as the others. ModelScan's 0% covers 80 of the 222 files; on the rest it gave no verdict. fickling reads pickle and pickle-based files only, and on those it rated 23 benign files LIKELY_OVERTLY_MALICIOUS and 67 more LIKELY_UNSAFE. That describes the configuration we tested, not every way fickling can be used.
What the default settings block
Flag rates are not the same as what fails your build. Each tool's default gate draws its own line, and on attack sets other people wrote, the trade-off is stark. ModelAudit and fickling block every PickleCloak attack file, and also 41% and 96% of benign files. Hayward blocks no benign file, but at its default (HIGH and above) it lets through 41 of 97 PickleCloak chains. With --fail-on low it blocks 91 of them, still with no benign file blocked in this set. The full write-up has the table for all five tools.
Want more detail?This is the summary. The full write-up, Clean, or Just Not Alarmed?, has the method, how we grouped each tool's findings, the default-gate and coverage tables, and the limits.
What about Hugging Face's labels?
Hugging Face combines several scanners and shows safe, caution or unsafe from the results it has. "Caution" is a warning: usually its pickle check has seen an ordinary import it does not know, such as a scikit-learn model class. 46 of our 210 real models carried it, all normal files. "Unsafe" is the alarm, and we kept those out of the test on purpose. So: caution means look closer, not malware. Unsafe means do not load it.
We had the same problem
In August our own scanner raised warnings on 33% of a similar benign set, more than picklescan. We were quoting "zero false positives" anyway, because it raised no alarms. Today it warns on 3%. The August set was not identical, so part of that change may come from the files rather than the scanner.
On three attack sets other people wrote, counting its findings from LOW up, it now catches 35, 51 and 91 files, against 34, 49 and 91 in August. We have not held those sets back from our own development, so read that as a regression check, not a blind test.
What to take away
- Know what your gate blocks. Run your scanner on a sample of your own models, and on known attacks, at the setting you will actually use. Too loud creates triage work and invites bypasses; too quiet lets attacks through.
- Never record "no verdict" as clean. Fail closed, or require an explicit, audited exception.
- Check coverage against the formats you ship. A pickle-only scanner says nothing about your Keras or ONNX files.
- Match the tool to the job. A build gate and a by-hand investigation need different settings, and sometimes different tools.
- Ask any vendor: does the false-positive figure count warnings, what share of files got a verdict, and what does the default block?
The bottom line
All five scanners catch the textbook case in the harness, a pickle that calls os.system directly. They differ on files they are unsure about, on files they cannot read, and on where their default gate sits. An alarms-only false-positive rate shows none of the three. Treat any single "zero false positives" figure, including the one we used to quote, as half an answer.
Being honest about this
These numbers are self-measured until someone else reproduces them, and 222 files is a small sample. The harness, quickset, is public and MIT licensed, and the raw per-file results from this run are published beside it. The full write-up shows how to run it.
If your numbers differ from ours, write to hello@hedgerow.dev.