In his talk on Micro-scale retro-futurist anarcho syndicalism Brian Marick about the retro-futurism portion of his movement. He says that he was promissed a jetpack, and wonders where his jetpack now is. Some time ago, Alistair Cockburn made me aware of the Martin JetPack. Let’s turn this into a testing challenge.
Product: The Martin JetPack
Mission:
Test the Martin JetPack for any problems a daily user of it may run into. Check any usability problems, problems in functionality, and anything that could harm the health of the user. Please consider that we might want to get an approval from any public transportation law to use this.
Please leave comments to my blog or drop me a line.
Markus,
I love the idea of testing challenges from different bloggers and testing thought leaders. I also liked Matt Heusser’s recent test design challenge. Keep up the good work.
Here are links to my initial, rough, thoughts in the form of 23 Design of Experiments-based tests and some supplementary additional thoughts and questions.
My process:
I. TEST INPUT IDENTIFICATION
I first identified a set of test inputs associated with some trial runs (including different types of users; different patterns of flight; different external conditions such as weather conditions; different altitudes, etc.); and finally some additional testing thoughts to be explored further through experimentation or additional research). Those test inputs are shown here.
II. PRIORITIZATION (COMBINING TEST INPUTS TO ACHIEVE PAIRWISE COVERAGE IN AS FEW TESTS AS POSSIBLE)
Next, because those test inputs would create more than 2.3 billion possible tests, I used Hexawise, our test case design tool, to generate a set of high priority tests to start with. Since there is strong evidence that the significant majority of defects (often ~85%) can be triggered by just one or two test inputs, we start with the smallest possible set of tests that will test every pair of test inputs together in at least one test case. We can accomplish this (relatively thorough)* level of coverage in just 23 test cases in this instance based on the test inputs I have identified. It turns out that only 91 tests are required from the 2.3 billion to achieve 100% coverage of all three-way combinations of test inputs. This would be extremely thorough* coverage.
The first eight tests of those 23 are shown
here.
If you or any of your readers want to improve upon my (no doubt highly imperfect and incomplete) draft sets of tests, please feel free to edit them. To do so, you can visit: http://app.hexawise.com/share/HYPFRD6N (Please note that a Hexawise account is required to edit the plan – free Hexawise accounts are available at http://hexawise.com/signup).
That link also points to additional testing ideas such investigating past lawsuits against jet pack manufacturers with Google searches and through Lexis searches. Those additional ideas and questions are found in the “Value Expansion” feature on the sidebar.
– Justin
*A note to put my “thoroughness” comments in context. There is a big garbage in / garbage out risk here that would be irresponsible to gloss over. Given that I know far less about jet packs than someone who has flown them, researched them, or fixed them, my set of draft test inputs are likely to be incomplete in extremely important respects. To have 100% coverage of all possible 6-way combinations would be phenomenally thorough by almost all standards IFF I had identified all the important test inputs and put them into the plan. Extraordinarily few defects would require 6 separate inputs to trigger. Even so, let’s assume for a minute that the jet pack stopped working at temperatures below freezing. In that case, if you neglected to test it in cold weather, the “thoroughness” stats (whether 2-way, 6-way or anything in between) would be meaningless. A catastrophic defect would have escaped testing.
Another (delightfully dangerous) idea: equip 4 testers with packs, put them on two teams and have them play a sport. This could be a modified game of 2 on 2 basketball or polo or soccer; exact specifics are relatively unimportant. Then have them try to maneuver with the jet packs to score goals.
While this approach would be a reasonably efficient way of finding subtle real-world maneuverability issues (“Gotta back up right away to avoid getting crushed by Jim who is hurtling towards me at an alarming speed…. Hmmm. Uh oh. It’s not quite as peppy maneuverable as I’d prefer it to be at this juncture….”), the broken bones to new insights uncovered ratio would be quite high. (… “Oops. @#$%!. Is that my shin bone peaking out of my jeans?”)
– Justin