An acceptable false negative rate really depends on the consequences of getting caught.
If someone's nefarious plan depends on smuggling a gun in, they want to be confident they won't be arrested or shot at the entrance. Even failing to detect 20% of firearms means there's an 80% chance they'll be caught before they can do whatever it is they plan on doing. This is also why it's important to have armed guards alongside the scanners. Scanners aren't very useful if the only armed person is the bad guy.
If the consequences of getting caught are negligible (as is the case for anyone trying to bring a box cutter through airport security), then the attacker can try as many times as they want without issue. Even if the false negative rate is low, they only have to get lucky once.
Annoyingly, I can't find any info about false positive/negative rates for various scanners. There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing.
The TSA's own security tests clearly show a significant percentage of guns, knives and explosives regularly get through. This is further confirmed by the number of travelers who, after arriving, discover the handgun they accidentally left in some pouch in their suitcase that was never detected.
> There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing.
Of course they are but the main reason there's no publicly available objective testing isn't only that sellers don't want it. In reality, no stakeholder in the security market wants it. The vast majority of high-volume public security like airports, concerts and sporting events is largely unnecessary and mostly ineffective but our current political/media environment requires appearing to "do something" to "make it safe". The Vice-President of "Make it (Seem) Safe" knows that their shareholders, politicians and the public aren't willing to pay more or be even more inconvenienced than they already are for 800% better "Make it (Seem) Safe"-ness.
Metaphorically speaking, the tiger repellent is working just fine, thank you. Those truly worried about tiger attacks feel safer and those being well-paid for preventing tiger attacks can claim virtually 100% effectiveness. So, if you start the world's best Tiger Repellent Testing Laboratory, you'll find a shocking lack of interest in buying your test data from both sellers and buyers in this brisk, profitable and growing market. Much like the lack of interest in objective testing data for lie detectors, astrology readings and placebo pills. The smaller minority of customers actually willing to pay more for improved detection (like Tel Aviv airport), do their own in-context performance testing anyway. In fact, a good proxy for doing your own effectiveness testing is available for free. Just look at what those under constant active threat with real consequences actually pay for and do.
I unknowingly transported ammo both to and from Mexico. I used an old backpack that I had previously used as a range bag from years ago. I ended up finding several 223 rounds in Mexico, then when I got back, even more 22lr.
Failing to detect 20% of firearms is probably a really deal. I would bet the majority of findings are just ordinary people who forgot to leave their gun in the car (akin to how TSA used to take grandma's knitting needles away every flight).
So, if your process really only detects people not trying to bypass it and people not even in the wrong then it's a problem.
Although I mean with the long lines at security you might as well just gun everybody down outside the stadium in that nice open area they are all packed into ...
If someone's nefarious plan depends on smuggling a gun in, they want to be confident they won't be arrested or shot at the entrance. Even failing to detect 20% of firearms means there's an 80% chance they'll be caught before they can do whatever it is they plan on doing. This is also why it's important to have armed guards alongside the scanners. Scanners aren't very useful if the only armed person is the bad guy.
If the consequences of getting caught are negligible (as is the case for anyone trying to bring a box cutter through airport security), then the attacker can try as many times as they want without issue. Even if the false negative rate is low, they only have to get lucky once.
Annoyingly, I can't find any info about false positive/negative rates for various scanners. There doesn't seem to be the equivalent of Consumer Reports or Underwriters Labs for scanners. My guess is that the numbers must be pretty bad if companies aren't willing to go through public 3rd party testing.