A while ago I put up a challenge for software testers. Here is the mission I used back then:

Product:
Regression Test Calculator

Mission:
Test the regression test calculator for any flaws you can find. You might gain bonus points, if you can find out how the calculation is done. Another set of bonus points if you can come up with a better approach.

In the meantime I found out that Ajay Balamurugadas actually found the link to the website, and sent it to James and Michael. I think he deserves some special kudos for this.

These are the responses I received.

Twitter

I recevied some feedback over twitter. Tim Western challenged the purpose of the tool completely. He was not alone with this opinion. Justin Rohrman, and Ben Simo felt the same about the tool.

Michael Bolton noticed that “boundary” was mispelled in a link to another tool.

Peter H-L pointed out that you can actually get minus coverage. I wonder what minus coverage should mean. Maybe having tested a product less than untested at all.

jss mentioned that I should try one test case that takes one tester two hours. Two testers can execute this test in one hour. I think that calculator didn’t know about Brooks’ Law very well. Or the calculator also thinks that carrying out a baby takes two women 4.5 months.

Blog comments

I recevied four blog comments from my readers. One was actually a pingback from Rasmus Koorits who submitted his report via email. I will get to this in the next section.

Phil Kirkham played a bit with the calculator, but lost interest after getting the first division by zero error.

Jean-Paul Varwijk submitted his report on playing for fifteen minutes with the calculator. He found some interesting problems in just 15 minutes. I asked him for a condensed report. I hope he will reply on that.

Emile Zwiggelaar also went for a 15 minute session, and also found some interesting problems.

For all three reports, go back to the original blog entry, and read their comments. They found some quite interesting stuff.

E-Mail

I received exactly one report by email. It came from Rasmus Koorits, as I mentioned earlier. He had seven pages full of interesting stuff that he found. The report is very thorough. He showed great passion in the craft. I don’t dare to put his report up here without having asked him beforehand. Maybe I will add this later. I think he deserves the most kudos for following up, and he did well on that.

Some endnotes

I intentionally didn’t mention too much details about the bugs and problems. I hope that some readers will later dive into the topic, and submit a report on their own.