A while ago, Elisabeth Hendrickson wrote a piece on endings and beginning. There was one sentence striking out to me:
I believe that the traditional software QA model is fundamentally and irretrievably broken.
I think there is something fundamentally broken in the way we are used to build and ship product. Triggered by Elisabeth I had an eye on stuff while starting to read The Psychology of Computer Programming only recently – yeah, I didn’t read all of Jerry’s books so far. Shame on me. These are my early raw thoughts on what might be broken.
To be honest, The Psychology of Computer Programming is more than 40 years old right now. So, what could you learn from a book that old? There is much in it, since it deals with the topic of psychology – a topic still that is still getting too few focus these days.
There, in chapter 4 on the Programming Group, I found a particular discussion about a topic which has been given few thoughts earlier these days. The effect is called Cognitive dissonance. What’s that? To cite wikipedia
Cognitive dissonance theory explains human behavior by positing that people have a bias to seek consonance between their expectations and reality.
That means that cognitive dissonance will make you biased towards believing whatever you want to believe. If you want to believe that the new car you bought was cheap, you will reject news about the price being lowered two weeks later.
What has cognitive dissonance to do with programming? Cognitive dissonance is the most often cited reason for claims for independent testing teams. IF the tester is part of the development team that created the product, then he will be biased to confirm the product is working.
Now, this is what the theory explains. Empirical data from agile teams with testers among them however, might point to a different conclusion. Sure, there are teams which would be better off with independent testers – probably off-shore in some other timezone across the planet. But that’s not my point.
Imagine, that at some point in the history of software delivery, we have been led astray by cognitive dissonance to form independent test teams. From that point on, the whole QA movement set off, leading to more and separation of testers and the remaining folks. At times, it seems like a whole caste of testers was created, which are less worth than those precious architects, designers, and programmers.
Recently, teams that overcame this second-class citizen testership since then have shown to be more productive with fewer errors. At least this makes me wonder how the world of software development today in the 21st century would look like, if we had found ways to overcome cognitive dissonance 50 years ago.
Sure, testers now-a-days have their role to play in software development. But what if this was just an evolution that moved us in the wrong direction? Think about it. How would software development today look, if programmers learned satisficing testing in the university, if they did pair testing of each others’ code – I saw a medical company doing that. Their first tester became that team’s ScrumMaster. And they are doing well.
I think it’s time to shake some of the foundational beliefs that we carry with us for decades now. Our ways to deal with cognitive dissonance and the call for independent testers is one of these foundations that I would like to challenge.
Regarding your point about an independence test team, there are pros and cons. The vital affect of this is that it reduces collaboration between teams and results in reduction of quality of the delivered software, increase in delivery time and cost. Remembering from Gojko’s keynote at last Testing and Finance conference in London, he reported a case of a development team he worked with which was shipping good quality software but had trouble coping with testing. They hired an experienced tester to help with the testing but the bug rate gone up instead of the obvious. This is as a result of developers becoming relaxed about testing and relied too much on tester to do the testing for them. The tester was overworked and could not cope. As soon as they realised what’s been happening they made him lead testing and developers to do the testing and the result was bug rate dropping sharply.
I think there aren’t easy answers how you split the test team. My experience shows that collaborative approach works well but it is not the cure for every situation. Bulk of system, integration, non-functional and UAT testing is done by independent teams outside the sprint anyway. The testing done within a sprint is fraction of overall testing. Overcoming biases is the hardest thing to achieve. You see what you want to see even if it is in front of you. I can count 50 biases that we all have and affect our testing ability in some ways. I will blog about it when I get down to doing it and approach friends like you for examples to support them.
Good questions, indeed. I often think that separation of programming and testing is an outcome of management that thinks the workers will loaf if you don’t set them against each other. When in a more generous mood, I wrote http://blog.gdinwiddie.com/2011/09/19/independent-interpretation/ on the topic.
I don’t think it’s because of loafing, but instead that the tester’s mind will be biased by the actual implementation. However, as your blog entry points out perfectly, it’s probably more about keeping an independent mind – and you can do that whether you worked together, or whether you read the same documentation.
I agree with your post & the independent mind Markus, but do you think it is easier for experienced Testers to keep this independent mind?
Possibly because they may have come from an independent test team (from cynical days gone by)?
I have seen inexperienced Testers succumb to cognitive dissonance & confirmation bias after a pairing with a Programmer – they were thinking that the “testing” they had just done was enough.
I found it took them longer to challenge & question what they were seeing in the pairing session.
What have you found on your travels?
The short answer to that is that I found no problems on this in places where testers were able – and had the freedom from blaming – to learn from their mistakes. There’s a saying in Germany: Some children have to put their hands on the hot plate – but they can only learn from it.
Thanks for the reply Markus – ensure there is a safe environment for learning, free from blame & where failure isn’t seen as negative.
You have put a smashy header at the top, but I wonder what the real idea of this post is. I see expressions of dubious feelings. If you claim that something is fundamentally broken, I would expect you really come up with a little more than counter assumptions against the current way of thinking).
Considering the header I thought, you would present results from studies of people working on behavioral economics / social sciences, but I have to learn that all that is mostly derived from a sentence of another blog post which also just expresses discomfort about the current situation and does not even clearly say what and why (and defers anything else possibly to a future “In the coming months, I might be writing here more. Or not. I’m not sure yet. We’ll just have to see how things go.” ).
Many DEV and QA teams have to face pressure, because companies feel they could outsource everything to low cost countries. I guess, there’s enough challenge for them already. So what are you shaking for?
I interpret your challenge as a religious (agile) motivated one.
My atheist view is:
Good software is mostly a matter of understanding what makes sense/does the job for the user. As the road to understanding is often paved with wrong assumptions and misinterpretations, people need to share information and have enough resources and time to interact. Whether DEV & QA people are in the same team has only little influence on this.
I dare to say that putting people physically into the same room will have more impact than being in the same team, but is still not enough.
IMO the most important factors are shared access to the information / resources required and the mindset of the involved parties. If people know whom to talk to (assuming that there are no additional/artificial/cultural barriers) and they are willing figure out what the software should do, it works. (Which does NOT mean, that this will be painless – in the contrary!)
It works even if they are in different teams. I see that every day with interacting globally spread scrum teams at work.
I think our biggest problem is our one-track-mind regarding understanding what’s really happening. We want simple solutions in complex situations (“just merge DEV and QA and all will be fine”) and jump on any correlation we like to believe in. Just like an ex boss of mine who avoided relational databases and booked the failures of any project using RDBMS he heard of on the technology.
IMO the same is true for the choice of merging DEV & QA teams or keeping them apart.
We need enlightenment here, not religion.