Indeed interesting. I wonder if there is a tool for generating input in a way that guarantees that all locations in the program are actually covered (in other words, that all reachable code has been reached).
Of course, this is no guarantee that the program actually works, but it would make me sleep better :)
I haven't had as much success with it, but it's so interesting that I'll keep trying. I'm also interested in KLEE, which I found in a similar HN story, but it has very specific build requirements:
KLEE is pretty cool. To compensate for the atrocious build instructions there's a docker image which contains KLEE built and ready to use (https://registry.hub.docker.com/u/kleeweb/klee/).
One type of testing (sorry, not a tool yet) which can help with this is combinatorial testing. Basically, when testing against a set of multiple inputs, create a test which will test every possible combination of input for a pair of inputs.
i.e. for inputs A, B, C, try every combination of A-B, A-C, and B-C
It sounds like a lot, but its not too onerous with the speed of computers we have today. This type of testing will flush out most bugs which will result from a particular combination of inputs, and is deterministic, when compared to fuzz testing.
This is not to detract from fuzz testing, just a note that it is effectively designed to be non-deterministic, which can result in a delay in bug detection.
Just to be clear, taking inputs in pairs does not guarantee complete coverage, even when trying all permutations of those pairs. I mention this because it is what the OP of this thread asked about.
