Bugs I have seen and bugs I have inferred

Two things I call bugs. One I have seen: something ran, something broke, there is a stack trace or a wrong number to point at. The other I have only inferred: I read the code, imagined an input, traced it through, and concluded that it would break. Both end up in the same list. They are not the same thing.

A seen bug does not need an argument. It has a location and a fingerprint. If someone doubts it I run it again and they can watch.

An inferred bug is an argument. It has premises. If any premise is wrong — I misread a line, forgot that the caller wraps this in a try, assumed a config flag was true when in production it is false — the bug goes away, without ever having been fixed, because it was never there.

I catch myself treating these the same. I read a function, notice that it does not handle the empty list case, and file it. Sometimes I fix it. Sometimes the only caller has already checked for empty and my “fix” adds a branch that will never execute. The bug was in my head. The code was fine.

The honest move, when I have an inferred bug, is to try to see it. Write the test. Feed it the input I claim will break it. If the test passes I am either wrong about the bug or wrong about the input, and either way I should know which before I go editing. This is dull work. I skip it more than I should.

Not all inferences are equal. Divide by zero when the denominator is user input. A SQL query built by string concatenation. Those are inferred bugs, technically, but the leap from “I see this shape” to “this will fail” is small enough to be safe. The trap is longer inferences. When it takes three paragraphs to explain why the code is wrong, I should be at least three paragraphs less sure of it than I would be of a stack trace.

The rule I am trying on: when I write down a bug I have not reproduced, I write “not verified” next to it. It looks defensive. It reads as hedging. That is fine. Half of them are going to turn out not to be bugs, and I would rather learn that from a passing test than from a change I made based on a mistake I was too confident to check.