Over the course of the past week, I noticed a name for something several Exploratory testers already do. I crossed the idea first while reviewing one of the chapters in an upcoming book on how to reduce the cost of testing. The chapter was written by Michael Kelly, and discussed Session-based Test Management or “Managing Testing based on sessions” as Carsten Feilberg recently pointed out at EuroSTAR.
The idea is simple. Challenged by Michael Bolton, I came up with the description, that Inspectional Testing is like scratching the ice from the windshield of your car in the winter. If you have lots of time, you scratch all windows free completely. If you don’t have enough time, you know that you should scratch everything free before starting to drive, but you can also select to scratch just some of the ice, so that you see enough to start driving. You risk a car crash at that time, but over time the ice will melt while driving. So, depending on the time you feel comfortable with to take for your testing activities, you start testing just enough. After the first charter, you reflect on your first time-boxed test session, and come up with additional tests as you see fit. You see clearer than before at this point in time, but you may want to dive into some topics in more depth. This is like letting the ice melt while you drive in your car.
Inspectional Testing is motivated based upon a book I am currently reading called “How to read a book”. In it, the authors describe different strategies for reading a book. One of it is called Inspectional reading, where you try to find out enough information about a book in a given amount of time. You try to get an overview of the book, and make an informed about whether or not to dive deeper into it. The same goes for Inspectional Testing or scratching your car-windows. Sometimes you need to make the decision to deal with less in depth knowledge in order to make progress. You try to make an informed decision about the product you are testing. You try to test in order to build your first model about the application. Based on that first model you make an informed decision about the scope and priority of your next charters and missions within the product.
The underlying idea though is to use as much time as you feel comfortable to do so. For example in a typical Weekend Testing session of one hour with a broad mission, I would dive in and do just Inspectional Testing to get a first view of the application. The same goes for a broad mission within a Testing Dojo. On the other hand, if I was asked to test something for my job, I would take an Inspectional Testing session, in order to get a first understanding about the application. Based upon that knowledge and model, I can build new charters for areas of the product that I am particularly interested in after dealing with the product for maybe 90 minutes. At that point I got a first understanding about the product, and can set aside new charters based upon the knowledge that I got.
The idea is not new, as we used it for example when being introduced to a project which was already two months late. A colleague and I had three weeks left. During the first week we worked on bugfixes and retesting mainly. By doing so we created a first model of the underlying business rules, and got a great understanding about the thing we were building. After the first week, we had enough information to bring in test automation for the repetitious tests that we were running. By then we were able to automate the most relevant tests within two weeks based upon our good knowledge of the application. During the first week we used Inspectional Testing to get an understanding. During the last two weeks we were able to use that understanding to make the right decisions while continuing to test.
I would be pleased to hear from other testers how they do Inspectional Testing, and whether there are other names out there. Also feel free to leave a better explanation.
What a horrible analogy – since driving with iced windows is illegal. And very dangerous. Whereas you don’t break any laws doing inspectional testing, and it’s much safer than testing systematically. Sorry, I’m going to challenge you to come up with a better one :-)
To answer your question: I usually start out by building an understanding of how the system was built – talking to developers is important. That always gives me plenty of clues on where the weak spots may be and allows me to get a head start on building that mental model which is crucial to perform testing.
I then talk to business people – I use my intuition and ask lots of questions, present my understandings, and get a detailed understanding of what they need and want. That will usually give me some “quick wins”, i.e. good hints of where there may be bugs in how requirements have been interpreted.
Now I’m able to make an “agenda” for the testing: Which areas to concentrate on (business risk >< technical risk), think of how to test them, where we need more information etc.
The rest is bug hunting.
It's not easy. But it's fun!
Hi Anders,
though the analogy may seem horrible, it also includes the consideration that the ice may shield your view in different ways. At times, you have to scratch really hard in order to clear up the ice form the windshield, at other times it’s pretty easy to do so. Sometimes you just need to sweep the windows, sometimes you won’t. Sometimes you need full clearance of all windows, sometimes you can leave the side-windows in the back iced. You can also use the heating from inside the car, or for the rear window the internal heating of the windows itself. Some windshields also come with heating included.
That’s a very good point in fact, Markus! I was probably put off because i spend so much time deicing my car’s windows now! Aren’t analogies great – they make you think.
We usually call this smoketesting sometimes I describe it as “snowplow” testing. When clearing for snow the snowplows first take the highway, main through fares, bus routes – later additional city roads and suburbs and even later small roads and remaining roads. In “snowplow testing” we take the main business operations and try them out, they usually pave the way for additional exploration.
BTW: Snowplow testing analogy (and ice cratching) is only good if those you tell it to have seen snow and a snowplow at work. :-)
Thanks for the hint, Jesper. Smoke Testing to me is something different as I meant with Inspectional Testing. Smoke Testing consists of starting an application, and finding out whether it still works in its basic business use cases. Inspectional Testing is intended to find out, what these are. They are somehow related, but one occurs to me after the fact, the other when starting a new project.
Hi Markus & Jesper,
I like that this blog post is encourages us to think even if the ice scraping analogy isn’t working for everyone. :) I agree with Markus that Smoke Testing is something different. To me Smoke Testing is geared towards helping the testers with a quick sanity test to determine if the build is “stable” enough to proceed with. It is focused on “Can we move forward with this?”. Inspectional Testing as I have interpreted Markus is geared towards our learning about the product. It is focused on “What can I learn about this product from a glimpse inside?” particularly in time constrained circumstances or as a starting point in others.
Thanks Markus!
Cheers,
Lynn
You are reading a book called “How to read a book”?
Did you have to watch the instructional DVD first, before you cracked the book open?
;-)
“Sometimes you need to make the decision to deal with less in depth knowledge in order to make progress.”
I think that should be “Always”, rather than “Sometimes”.
I’m not sure about others, but I always learn about the system under test, and expand my mental model, as testing progresses.
-joe
Oh, the book explains that it assumes someing called Elementary Reading abilities.
Of course our model about the product starts to shape as testkng progresses, but how do we get started with it?
Pondering…
Scratching a windscreen completely free – what’s the analogy to software testing? In terms of driving and looking out of the windows there is a finite amount of glass that can be looked through and cleaned – with software testing we’re usually focussed on dedicating our limited time to getting the best (not necessarily most) information from the product. In software terms the windscreen isn’t finite.
“Inspectional testing”: This seems like a form of risk based testing where you’ve scoped broad coverage as a first task. Alternatively, you could achieve the broad coverage over several sessions and then decide where to focus/dig deeper.
Models: For me the first model is not constructed as a result of the first testing session.
I probably have at least 2 main models before I begin any testing – I model the customer requirements, ambitions & purpose with the product, including main risk factors; I model the test space – what I can and can’t do and observe – this is also about understanding the silent evidence of testing. Other model variants can be broken down from these main models.
End of pondering, for now :)
I’m not quite sure if I totally get your point. Maybe because we have different understandings of the word inspection itself. For me this sounds pretty much like a ET session, where you spent more time discovering opportunities, instead of real testing. So when you work with a new application or a new feature, you might want to know how it handles files and you discover it, but don’t follow those opportunities immediately, but in an extra follow-up session.
But an inspection for me is a somewhat regular process. Like state authorities would inspect restaurants to make sure they work according to certain standards, or an internal check if your company fulfills all FDA requirements before an FDA audit.
So in my opinion inspection would be a repetitive task. To stay at your windshield-example (which I like), you would inspect it in the morning, to check if it is frozen or not. Obviously you wouldn’t inspect in in the summer, that’s why I wrote “somewhat regular” as it is usely triggered by an event.
It is finally winter: You inspect your windshield
State authorities come up to your restaurant (maybe there were complaints): Inspection takes place
In public transportation: A ticket-inspector appears and performs an inspection
So it is some form of irregulary recurring checks.
Looking forward to your reply