Monday, April 28, 2014

Criteria to fix a bug

Ok. So this may seem stupid, but I was pouring over a list of defects in our product backlog the other day and I started thinking "How do we really decide if a bug should be fixed?" Everybody talks about severity, and I suppose that is the most important factor in most cases; but that's not the only factor.

It dawned on me that there is a simple equation to represent the value of fixing a bug: The value of fixing a bug is the ratio of the impact of the bug to the cost of fixing the bug.

valueOfFixingBug = impactOfBug / costToFixBug

The higher the impact in relation to the cost, the more value there is in fixing the bug.

It doesn't seem like I'm saying anything novel, but in all my years of software development it just seems like people don't really think about *impact* -- they think about *severity* -- and the cost part of the equation doesn't get talked about all that much. Maybe because clients don't consume themselves too much thinking about the cost of defects, because they get them for "free", not considering things like opportunity cost.

Let me try to make this a bit more useful...

The impact of a bug is *not* the same thing as severity -- at least not the way most people define it. Here are the things I would consider the factors of impact:

1. When it happens, how bad is it?
2. How often does it happen?
3. Who does it happen to?

These are the things that, as they increase in magnitude, should sway you in favor of fixing a bug or prioritizing higher than other work.

The first item in the list ("how bad is it?") is what most people call "severity". This is a concept that exists in most bug-tracking tools, expressed as a number (1-5) or a word that conveys the severity. And we use phrases like "This is a sev 1" when we talk about them.

The "how often" and "who" parts of impact are not discussed much. It might be useful to combine them with the severity to come up with an "impact".

And so of course we can do a similar expansion of the equation's denominator (costToFixBug). Here are the things I would consider the factors in cost:

1. The cost to determine the source of the problem.
2. The cost to implement a fix.
3. The risk of breaking something else.

These are the things that, as they increase in magnitude, should sway you against fixing a bug or prioritizing it lower than other work.

I thought it was mildly interesting to think about this problem in this way.