In the past there have been multiple folks proposing a rename of the famous testing quadrants that Brian Marick wrote up a couple of years ago. Lisa Crispin and Janet Gregory labeled them Q1, Q2, Q3, and Q4 in their Agile Testing book. That naming misleads some folks as they think about a progression in the four quadrants. I recently read something that triggered a thought. I think we should rename these quadrant completely, and drop the counting digit at all. While several other folks have tried to find new names for the quadrants, I came up with a different thought: let’s try to find person names for the quadrants. Here is my first proposal for those.
Technology-facing team support
In this area fall tests that support the team. These are low-level unit tests, as well as some higher-level unit tests that combine for example two different classes that need to work together. GeePaw Hill made me aware when copy-editing my chapter for How to reduce the cost of software testing that these are usually called micro-level tests. These tests shall ensure that the team is building the product right.
This quadrant is a lot developer-centric. Let’s call this quadrant Kent. Kent will develop with other programmers a lot, usually working with TDD and lots of pair programming. Kent will help others, and heavily automate every single design decision that he made so that the whole team is able to remember the choices from today, and have a unintended change detector running in their continuous integration build.
Business-facing team support
Business-facing tests that support the team consist of story level tests. Usually, when using ATDD, these are fully automate during the iteration. These tests shall help the team to see whether they build the right product. Also workflow, use case, and scenario tests fall into this quadrant.
This quadrant combines the business viewpoint on the application together with developer practices. Let’s call this quadrant Ward. Ward will work closely together with business stakeholders, using examples to understand the underlying business rules, and automate them while working on the features. Ward will also include these automated tests into the CI build, and the team heavily relies on these tests while working on the system. At times, Ward works from the examples into the architecture and design of the software.
Business-facing product critique
This quadrant consists of manual and user acceptance tests. These are usually carried out by sapient testers. The tester evaluates the product, and checks for problems in the application – something that does not seem right according to a couple of heuristics that he has in mind.
Tests in this quadrant are usually exercised using an exploratory approach. Let’s call this quadrant James. James will work on short time-boxed sessions of exploratory testing. He has the mission of the team in mind when he exercises the application, and finds crucial problems in the product before it is shipped to the customer. He provides a session charter for each of these sessions, so that he knows when he got too distracted from what he actually wanted to achieve. James also closes the Inspect & Adapt cycle by doing a debriefing with his session notes after each time-boxed session, and reflects with a team colleague about necessary adaptions to the current approach and testing priorities.
Technology-facing product critique
Tests in this quadrant aim at the so called non-functional requirements. Usually these involve special tests like performance, load, and stability tests. The system is exercised under concurrent situation, and needs to prevail the intended target user load.
Tests in this quadrant require special knowledge. Let’s call this quadrant Scott. Scott knows all the performance tweaks that you can take for an application. He know how to fire up load tests that can bring down the system in seconds, and he knows the limitation of different usage loads, and how to reproduce them in a test harness. With his tools, he has super powers that frightens every DevOp, and system operator.
Kent, Ward, James, and Scott
I think with these names, the questions about a good testing strategy on an agile project shifts from fulfilling all the different quadrants – and usually forgetting Q4 tests – towards the different names. Instead of asking how many tests we do in Q1 of the quadrant model, we can ask how much influence Kent should have on our tests. We also start to think whether Ward and James should have equal influence on the overall testing strategy, or whether you want to have a preference for either of them. Oh, and we certainly won’t forget Scott, since Scott needs a whole lot of attention.
We can also shift our conversation when it comes to staffing. Rather than thinking about particular persons, we can ask whether we have enough Scotts on our team. We will also certainly notice whenever we lack some of Ward’s skills, and can directly hire for a person that brings that additional knowledge in.
If you struggle to remember those names, let me reveal my motivations. Since Kent Beck wrote the dominating book on test-driven development, I decided to give the technology-facing quadrant that supports the team the name Kent. In the business-facing quadrant for team support, I put Ward Cunningham, the inventor of the Framework for Integrated Tests, effectively one of the first frameworks to support ATDD. For the quadrant with business-facing tests that critique the product, I was struggling between Cem Kaner, Michael Bolton, and James Bach. I picked James here, since he wrote the most about session-based test management, and Exploratory Testing. Cem and Michael would have been an equal pick, and it was a hard decision. For the technology-facing tests that critique the product, I picked Scott Barber, performance test specialist.
I think these names have the power to change the conversation from a ranking between Q1, Q2, Q3, and Q4 towards the right balance between these four gentlemen.
The only flaw I see in the model stems from the realization that I only picked male names. Reflecting over it, what would be appropriate female pendants? Probably Emily, Liz, Elisabeth, and Mieke. Can you guess which ladies those would be?
I’ve been using the quadrants as a model quite a bit recently, and this is an interesting view. I’ve developed two challenges to the traditional (or original?) view of the quadrants that I’ll throw at you here (since I’ve been too busy to blog lately).
The first observation is that I don’t see clear buckets between the quadrants – particularly between the top and bottom halves. In other words, there’s a lot of overlap between Kent and Ward, and between Scott and James. There’s a fine line between unit, component, functional, and acceptance tests (with the similarity that all tests run fairly quickly and all either pass or fail). Similarly, some ilities can live with Scott, while some may live with James. Furthermore, a single ility can span their worlds (e.g. Scott may write stress tests, and James may explore for crashes).
The second observation (that I really need to blog about soon) is that James’ work doesn’t have to be (entirely) manual. I know that Lisa and Janet labeled that quadrant as manual, but in practice, I’m finding that I can cover that quadrant extremely well through data analysis (assuming it’s a program / feature that real customers can use).
Although I’m a fan of the quadrants, I can’t help but apply the George Box quote about models.
Interesting idea! Particularly the women’s names!
It’s true that numbering quadrants made some people think that was the order in which they should be done.
We’ve actually moved away from Brian’s term “support the team” because some people didn’t understand what that meant. We’re leaning towards “guiding development” or something similar that reflects these tests are helping us know what code to write.
It’s great that many people are re-thinking or extending the quadrants, as we did with Brian’s original “agile testing matrix”. It might be a good idea for each team to create their own quadrants, with terminology that’s meaningful to them, and a taxonomy that applies in their context.
Thanks Markus for provoking more thought on this!
Clarification – Brian numbered them. Lisa and I tried a bunch of different numbering schemes and none of them made any more sense so we left them as he had them.
Like Lisa, I like that people are challenging the quadrants or any model. In “More Agile Testing” (our new book), we have included some other’s ideas to show that it is good to challenge – that is how new ideas are born.
One thing that concerns me with the names is that people will associate only that type of testing with the quadrants. For example, I think there is actually more to Q1 than unit tests, although that is probably the most known / used type of testing. For example, I might consider running connectivity tests as part of Q1. And Q3 – so much more than exploratory testing, although it is a big part.
Like always, your posts make me think, which is good.
The model is wrongheaded. It has become a symbol for how the Agile community misunderstands testing. I’d like to see someone critically analyze this once in a while. I was nearby when Brian and Cem came up with this. I thought it was harmless at the time, but I have come to feel that it helps preserve and protect testing ignorance– because the two distinctions it tries to make actually don’t exist.
When it comes to models, I keep in mind that eventually all models are wrong, and creating a perfect model usually is a waste of time as it would have to have so many corner cases that it becomes useless as a thinking aid.
That said, thinking aids should be applied while thinking, not replacing them.