We have a bug in our industry: We over glorify bugs. We love them so much, that we use them as a primary metric for quality.
My take is that that is a bad idea. Bad, as in driving undesired behaviors that are misaligned with our desires to improve quality.
Let’s begin with an analogy, and then move to modeling this on software.
A tale of two restaurants
Joe was hungry. He was on a business trip, and it was lunchtime, and he just entered Bologna, an Italian restaurant.
It took almost 3 minutes before a host led him to a table, which was set up simply, yet invitingly. There were some loud voices from the kitchen, and Joe wondered what they are arguing about. When the waitress came back he ordered a pizza, which he noticed was recommended in reviews on TripAdvisor. While he was waiting for his order, he saw the staff going around customers – acting nice, but not overly nice, smiling from time to time, but not overly eager.
Then the pizza landed on his table. It smelled just gorgeous, and tasted even better. The pizza base was perfect, rich in flavor, as if the flour was different from any other he tasted. And the topping – was amazing. It was dead simple – generous slices of mozzarella cheese over a delightfully delicate tomato sauce, and topped by fresh arugula leaves. That pizza reminded him of some long-ago childhood memory that he couldn’t put his finger on, and he wished that pizza will never finish.
As he walked out he noticed that paint was pilling off the door frame, and a couple of tarnished floor tiles.
He was feeling happy, and ready for the rest of his working day.
The next day Joe decided try Pizza World down the road from Bologna, which was also recommended on Tripadvisor.
He was greeted with a big smile by the host, who immediately showed him to his table, and left an open menu for him to browse. He decided to order a mozzarella pizza, following yesterday’s success. As soon as he put the menu back on the table, a waiter came over, smiling friendlily, and took his order. While he was waiting he couldn’t help noticing the speckless while-tiled walls, and shining counters and the squeaky-clean glass-wall looking over the preparation area in the kitchen.
Shortly after, the pizza arrived, making Joe almost drool with anticipation.
And the pizza was – oh, bland. Regular. Nothing out of the ordinary. Don’t get me wrong – it was made to perfection, just that it tasted good, and that’s it. No childhood memories, no superlatives to offer, no sense of revelation.
A week later his colleague texted him from the same town, asking for a recommendation for lunch. Guess which one Joe recommended?
Don’t get me wrong here – I am not saying – make bugs. Assign X% of your time for bug making, because your customers will love you for that. Of course not!
Still, the first restaurant is more “buggy”, and yet has a superior product compared to the second one. So using a zero-bugs as a metric means next to nothing. Having policies that help us avoid bugs is a healthier approach.
Why not use bugs as a metric?
Our obsession with bugs brings on some really cargo-cult behaviors with it. Instead, I propose to ignore metrics on bugs. Here’s why:
- “Life is what you focus on” (I didn’t find the origin of this quote). If we focus on happiness, we get increased happiness; if we focus on strength, we get more strength; if we focus on bugs, we get, well, more bugs!
- Let’s focus on what quality is, not what it is not:
If quality is delivering on time, let’s focus on cycle times, to improve our predictability;
If quality is happy customers, let’s focus on customer satisfaction, to improve our feedback from them;
If quality is testing, let’s focus on testing metrics like coverage and testing feedback times and early impact of tests;
If quality means good software, let’s focus on excellent engineering practices that make good software;
You get the picture. - Old habits die hard. And changing culture is even harder. Bug worship mentality has been around for many years, and it will take perseverance and intention and discipline to change ourselves and to influence others.
- Bugs are not all that bad. Software is a highly complex practice, and it only makes sense that we make mistakes now and then. It happens on roads, it happens on healthcare and there is no reason to think we can get rid of all bugs in software.
Note, that the Bologna restaurant from the story above has no bugs on its core service – the food, but allows few bugs on the periphery. You must decide what is good enough in your context, and what you are happy not to fix. - Having said that, bugs are not the problem. The real problem is what makes it easy to create new bugs.
We are much better off creating the conditions in which it is harder to produce bugs, than investing in a detect-fix vicious cycle. The latter would be like blaming all road accidents on drivers. The former will be like investing more in safer roads and cars. - Our worst sin is that the remedy is out there. The knowledge exists, yet many of us are inapt to practice it because we have the wrong assumptions and knowledge and experience.
Let’s stop talking about bugs as if we can make them go away for ever. We can’t. Instead, let’s turn our focus to finding what are the desirable outcomes in our specific context, and then choose the metrics for the specific point in time to get us just enough to be able to choose the next one. My hypothesis is that by measuring the right things in the right time, bugs will stop bothering us that much.
Image by Clicker-Free-Vector-Image and Digital_Designer on pixabay
