The Deliberate Tester – Chapter 4: The Challenge

Back in 2011, I approached Rob Lambert at the Software Testing Club on a small series, packed into a narrative format as I wanted to try that out. Rob decided to run that series on the Software Testing Club back then, and I had some fun writing it. Skip forward 11 years, and the Software Testing Club no longer exists, it’s been a while since I have been in touch with Rob, yet I figured, let’s see how this series aged over the years. As a sort of throwback Friday for myself, I will publish the entries on a weekly basis, and read along with you. I think I ended up with eight chapters in the end and might add a reflection overall at the end. In case you want to catch up with the previous parts, I published these ones earlier:

Chapter 4: The Challenge

“Peter, you have done very well over the course of the last week.”
Peter sat in Eric’s office and was wondering what his boss would tell him after the first week of his first job after leaving the university.
“I would like to give you the assignment to do on your own. A challenge.”
“Ok, so no pairing?”
“Exactly. Considering your progress in the past week, by now you should know the basics in order to succeed at it.”
“Well, thank you for your confidence. I’m not that sure about it, yet. So, what is this challenge all about?”
“I want you to test a piece of software. You may use anything for the testing as you may see fit. You get two weeks for your preparations, planning, infrastructure, execution, and final report. In the end, I would like you to present your final report in a brown bag meeting to the whole team. It’s a tiny piece of a larger application suited for this kind of challenge. It’s a basic login screen for some web application.”
“Oh, that sounds great. When may I start?”
“You may start right away next week. In case you need any help, you can ask Jennifer and John. I planned for them to support you on this accordingly. I will send you the details about the application to test in a mail providing you with the documentation as well. Ok?”
“I can’t wait to get it started!”
“Great, then have a nice weekend.”

When Peter got back to the office on Monday, he opened the email from Eric and started to dive directly into the application. Eric had provided him with a URL to the application. As he started the browser, Peter was already thinking about the first test cases to exercise. Peter quickly took a piece of paper and set up the charter for the morning.
“How’s that application coming along?” Jennifer asked.
“Oh, I’m just setting up a charter to get to know the product. Exploring the product in order to make informed decisions about the particular test approach seems a good starting point to me.”
“Sure. But you should keep an eye for traps.”
“Traps?”
“Yes. In all of our testing, there is a high probability to fall for traps.”
“Traps…. hmmm, do you have an example?”
“Do you remember the bug in the UI we had last week?”
“Yes, I worked on the automation code for the build the day before. On that day I learned, that automated tests are not sufficient in themselves. We need to explore the product also.”
“Exactly. We pinpointed the problem in that session. We started follow‑up testing so that we could give the developers as much information as possible in order to have them fix the problem. Since the problem was familiar to me, we could track it down in no time. But, what could have happened if the problem was not that easily tracked down?”
“Oh, you’re right. We might have spent all day to find the problem. Thereby we would have ignored our charter while spending time on pinpointing rather than testing. So, that would be a trap?”
“Yes. There are many more out there. I don’t want to spoil you, though, rather I can’t. Just keep yourself aware. As every tester may fall for a different trap, over time you will need to find out how you get fooled yourself ‑ and prepare accordingly for it.”
“Thanks for the advice. I will keep that in mind.”

After 90 minutes following his charter, Peter felt uncomfortable with the state of the login functionality. He had come across a whole lot of bugs, his session report was full of problems. Therefore he decided to talk to Eric about his first exploratory session.
“Do you have a minute?”
“What’s up, Peter?”
“Regarding the challenge you gave me… Well, I explored the functionality this morning. It seems to be very buggy. Here, this is my session report.”
“Ok, can you give me a quick five-minute summary?”
“After having spent nearly all of the session trying usernames and passwords, there are serious problems in the interface. First off, when I enter the password field and start to type, the username field gets cleared. I can work around that issue by typing in the password first, but which user would do this? Second, it does not matter whether I enter a valid or an invalid username/password combination, they all successful login. And…”
“Alright. Did you talk to the developer? The login screen is actually a new functionality, which was started just today. I gave you the URL of the staging server, where the latest stable build is put on.”
“Oh, this is just developed right now?”
“Yes, April is the new programming apprentice. She got the challenge to develop a login mechanism.”
“Now I understand. You put us into two related challenges.”
“Yes, exactly. Ok, let me introduce you to April. I seem to have forgotten this one.”

As Peter and Eric arrived at April’s office, she just finished submitting her last changes.
“April, we just noticed that we didn’t introduce you to Peter. Peter is a new employee working as a tester on your challenge.”
“Hi Peter, nice to meet you. So, you’re testing the login screen then? I just pushed a new build to the staging server.”
“Oh, great. I spent some time this morning exercising the login screen on staging. I thought it was a production login. I noticed some areas, which could need some improvements. Shall we go through my session report together?”
“Oh, sure. Maybe we should sit together and coordinate our efforts. I’m not quite finished with it, but if you’re already testing it, you should at least know on which parts of the screen I’m going to work next, and which need improvements. Let me just finish my next Pomodoro off, then we can sit together in about half an hour.”
“Ok, that’s fine with me. Thereby I should be able to get some information from you on how to test the screen, whether we need regression capabilities using automated tests, and which areas are probably risky. See you in half an hour then.”

During that day, April and Peter made great progress. In the evening Eric visited them for a debrief.
“You’re still at Peter’s desk. It’s a great thing to see you two collaborating.”
“Yeah, we reached a common understanding of how to improve the login screen and how to test it.”
“Outstanding. Now, you have a plan for the next two weeks. April, what did you learn today?”
“Well, first I started to develop on my own without knowing what I was up to. Peter talked me through some alternatives, using familiar products he got to know over the internet. We agreed on some layout and functionality improvements. This has been fun.”
“Yes, exactly. It’s the same for me, too. At first, I was skeptical; April could bias my testing. But by getting our heads together, we were way more productive than on our own. We even agreed to pair program and pair test for the rest of our challenges.”
“Wonderful. I hope to see you two tomorrow for the debrief, checking about your progress. Keep up the good work.”