Yesterday I mentioned something on twitter, which I would like to give some more thought in a blog post here. While picking a name for for this blog post, it occurred to me that there is a fallacy lurking behing that phrase, which I deliberately call the Testing equals Quality fallacy.
That said, here is my quote from yesterday:
There are three parts in this sentence. First, we’ll look at testing and why it does not bring in quality to the product in itself. Second, we’ll see what testing can provide, if it does not provide quality in itself. Since there are many experts claiming that testing is not optional – which I fully support – there must be something to it that makes it worthwhile despite the bringing quality to the product. Last, we will dive into the reaction bit of the sentence, and what this means for testing.
Testing does not automatically yield Quality
What does a tester do on a full day of his testing job? Now, obviously she tests. But how does she do that? She creates test cases, test models, applies heuristics and oracles in order to decide whether an observed behavior is a problem in the software, or whether it is not. Testers attend bug triaging meetings, specification meetings, walk over to the programmer, and write reports about their activities. But does all of this create Quality in the product? Maybe. If your tester fixes the bugs she finds right away in the software. If your tester sends programmers to programming training. If your tester negotiates contract details with your customer. If your tester rejects ambiguous and vague requirements and user stories before they make it into the development cycle. But do your testers do all of this? Do they decide about all these factors? Or is it maybe someone else? Now, think about it this way, what provides more quality than one of the things I mentioned? And how do you expect your tester builds in the quality into the product without doing any of these?
What does Testing provide, if not Quality?
But, then you ask me, what does testing provide, if it’s not for the quality? Well, when testing a software product you gain information about the product, about its state, about its problems, about its itches, about its flaws. Testing provides information about the usefulness of the software, about whether the software solves the problem that the customer had before asking for it. Testing can inform about the product, the readiness to ship it, and about possible side-effects of making the decision to ship. Testing helps to see the blind-spots of the programming work, and helps to make the status of the quality transparent. Testing provides information about the quality of the product. That’s it. Information.
What does provide Quality after all?
But, wait, what does provide Quality in our development process? There must be something? Maybe it’s Scrum that provides Quality? Or XP? Or Kanban? Maybe the Waterfall? No. None of the mentioned do provide quality in themselves either. The reaction to the information you gather by testing the product can help to provide quality. It’s that simple. But let’s look at the implications here. The reaction is a human part in the system of effects. Based on your reaction to the information provided by testing different people will do different things. When you start to blame someone, he might hide this information from you the next time. When you placate your customer pushing more features into the development cycle than your team is capable of delivering, the information that testing can provide might just scratch the surface. In combination with blaming the team this may even yield dysfunctional information hiding politics in which you discover problems in the software only until it’s too late to react upon. Sounds familiar? I hope not.
Having a human reaction in the system implies that there is a human who is reacting, which means that a human needs to take the decision to react in a certain way to the information about the product. As taught by Jerry Weinberg in QSM Vol. 1
Whenever there’s a human decision point in the system, it’s not the event that determines the next event, but someone’s reaction to that event.
Decision By People (page 111)
Since the reaction to the information is based upon a human decision which directly may influence the quality of the software, we build a feedback loop. Human decisions about quality are always political and emotional, but people will claim to be rational about it. Quality decisions are political since they influence the development team in the same way as the customer. There may be reasons to ship the product earlier, and make money earlier, maybe just for the sake of attracting a new market, or to claim to be capable of delivering a new superior product and get the market share. Yet, decisions about quality are emotional, since they involve someone deciding about how the reaction will appear to others.
So, next time when talking about testing and quality, remember that testing might not provide quality, but reaction to testing, and to the information provided by it does directly influence the quality of the product.