Santhosh Tuppad sent around a little challenge for testers a few weeks ago. When I read through it, I couldn’t resist the temptation to the challenge, even though I don’t want to win that challenge; just to sharpen my saw at it. Here we go.

What if you click on something (A hyperlink) and to process or navigate to that webpage you need to be signed in? Currently, you are not signed in. Should you be taken to Sign up form or Sign in form? What is the better solution that you can provide?

Why would I need to sign in, anyways? Is it for the sake of getting my email adress to submit some more newsletters to my spam folder? Or is there some sort of cohort metrics running in the background? What other kind of information can I get from the webpage as a first time visitor? If this takes me longer than 10 seconds to grasp, I am no longer interested.

A better alternative would be a preview – like the tiny things I do with the “Continue reading” link on my blog. If I get the appetizer, and have become attracted by it, I might be more willing to join your sign up service, anyways. If I get two links to sign up and sign in I will be more pleased when I already have an account later.

Using “Close” naming convention to go back to the homepage is good or it should be named as “Cancel” or it is not really required because there is a “Home” link which is accessible. What are your thoughts?

I feel that I don’t have enough contextual information to decide about “good” or what “should” happen. Provided, I want to give you a satisficing answer to your question, here are some of the questions, that I want to ask you:

• Is the naming convention followed on the rest of the web sites? If no, then my answer does not really help you at all. Fix the inconsistency in your product first, maybe.
• What kind of sub-dialog is closed when I hit it? If it is a preference screen, then close could indicate to save the changes to the preferences made. Then I would prefer something like “save” and “cancel”, if the changes are not applied directly (like on a Mac). If I cancel a purchase of a 30,000 USD item, I would like to get this benefit as soon as possible, that I will actually not purchase the item.

Logout should be placed on top right hand side? What if it is on the top left hand side or in the left hand sidebar which is menu widget like “My Profile”, “Change Password” etc. – Is it a problem or what is your thought process?

I don’t know. Let’s schedule a usability testing session, or at least add some tracking to this function to get the feedback whether people actually find it, or whether logout happens most of the time through implicit cookie deletions after the cookie holding time expired.

Current design of forgot password asks for username and security answer and then sends a link to e-mail inbox to set new password. How does “security answer” increase the cost of operations? Also, what questions do you frame for security questions?

What is our target audience? In case most of our users are expert IT users, we should come up with other questions for the password lost security question. If we have to deal with kindergarten kids, we should maybe come up with another set of questions, but less hard to answer.

Regarding the cost of operations, we will have to deal with people who lost their passwords, but also forgot their security answers. By then we will have more trouble in our support department. On the other we will also eventually end up with fewer misuse of accounts – but the security question might also provide a source of distracting our actual knowledge. It could be that we are too biased to rely on our security question that we don’t see how many accounts are actually abused by too easy to answer “security” questions. Thereby we may loose many customers and our own reputation if the press gets a hold on it.

If you had to design “Forgot Password” working, how would you do it and why? You are free to give different many functional designs.

I would prefer sending out an email to the account owner, with a link to reset their password. To avoid abuse of the account, I would also add old password, new password twice validation for the changes to your password, and I would also add extra security for changing your email adress.

Why? As a user of multiple internet systems, I find this type of resetting my accounts’ passwords most comfortable. Sure, I will have to remember that I need to change my email adress when once becomes obsolete (e.g. when changing jobs), but in the past few years such changes have become rare for me.

There is neither account lockout policy nor captcha for the login or security answer forms; what kind of problems do you see with the current implementation and what do you propose?

As I am still not sure why we would need a login function at all, I will ask a lot of questions, first. The answers might guide me to some of the following answers. If it’s an online gaming community where you can chat with others, but nothing serious will get harmed (like stats earned in games), there might be a confusion when someone else signs up as myself. If it’s an online banking system, then someone else might steal my money. If it’s an online shopping platform, then I might order a lot of stuff, but that will never get shipped to me (by then I might know where I could look, actually). If it’s my own blog, then too bad, I will start another one. :)

Well, it is about context and there are no best practices in general. What are your thoughts on usage of captcha? Where should they be used and why?

Captchas should be used where useful. If I maintain a web side where viewing impaired are exchanging their thoughts, then captchas might turn out contra-productive. If my web page theme does interfere with the proper display of the captcha, then I should change something, either the theme, or leave out captchas. If I want a lot of flooding users prevent to put spam on my blog, I should put that up on my blog. If captchas interfere with my blogs mobile theme, then I should disable them. (That actually happened on my German blog (and also here) – hint to plug-in developers for WP Touch and Spam Free WordPress!).

If you are the solution architect for a retail website which has to be developed; what kind of questions would you ask with respect to “Scalability” purpose with respect to “Technology” being used for the website?

• How many customer do you expect in the first month after release? How many after one year?
• How many items will be purchased by the customers on average? What about minimal and maximal numbers that could give us hints to the standard deviation?
• What’s your value and growth hypothesis for your product?
• How reliable should the page be?
• How much downtime can we tolerate?
• What other kind of data do you have?

How do you think “Deactivate Account” should work functionally keeping in mind about “Usability” & “Security” quality criteria?

“Deactivate Account” should inform me what will happen next, and ask for my confirmation about this step. For a limited period I want to be able to re-activate the account, if I accidentally hit the confirm-button, but after 30 days (or so) it will be my fault. (That actually happened to my YouTube account a while ago.) I might become frustrated about it, or I might search for another function – but that will be ok if I take the decision to deactivate my account. I know the risk. I can handle it. After my account is lot, I don’t want other users to open an account using the same name, though. Otherwise an old link might refer to a porno video, or something like that.

For every registration, there is an e-mail sent with activation link. Once this activation link is used account is activated and a “Welcome E-mail” is sent to the end-users e-mail inbox. Now, list down the test ideas which could result in spamming if specific tests are not done.

For every registration? What about emails that already are in the system? What about constructed emails like abc+test+test+test@gmail.com, if abc@gmail.com is already in the system? What happens if already activated accounts don’t get a new mail, but I use the same email twice for registration, but without activation? What happens I if I activate an already activated account accidentally by clicking the link twice? Are invalid email adresses also handled?

In what different ways can you use “Tamper Data” add-on from “Mozilla Firefox” web browser? If you have not used it till date then how about exploring it and using it; then you can share your experience here.

Taking a look on the plugin page, I saw that I can modify http requests with it. I can use the plugin for analyzing vulnerabilities to web forms and http/https pages in general. I can explore whether a particular responder also responds to different requests thereby exposing critical functionality to url hackers. I don’t want to start Firefox right now to explore the plugin, though. :)

Application is being launched in a month from now and management has decided not to test for “Usability” or there are no testers in the team who can perform it and it is a web application. What is your take on this?

I inform management about the risks I see in this decision. If there are none, I will probably not say a word. If I am deeply concerned, I will work hard on delivering my viewpoint, but avoid to “educate” management too deeply about their decision. I will make my choice transparent though, and refuse any “who tested this piece of crap?” questions later.

Share your experience wherein; the developer did not accept security vulnerability and you did great bug advocacy to prove that it is a bug and finally it was fixed. Even if it was not fixed then please let me know about what was the bug and how did you do bug advocacy without revealing the application / company details.

I once worked on a use case where money was spread from one parent account to a bunch of other accounts. There were multiple business rules involved in this spread. We worked with test automation to overcome the simpler business rules. We had expressed our test data in a readable format. From this format we extracted the vulnerabilities for our bug reports.

One example had to do with two different account types. There was account type ABC and type DEF. DEF should get money first, but only get it’s special services activated if there was sufficient money left. We constructed a test case in FitNesse, that could exemplify all the conditions easily. We marked down the preconditions for the test (account of type DEF, account of type ABC, parent account with some money), then executed some function (spread money up to 1234 USD), and checked the post results (money in the parent account, money in ABC and DEF, states of the accounts, and the special services). We also documented our expectations in between these pure test data. Convincing our programmers and chief programmers to fix bugs as they occurred was very easy with the examples provided.

What do you have in your tester’s toolkit? Name at least 10 such tools or utilities. Please do not list like QTP, LoadRunner, SilkTest and such things. Something which you have discovered (Example: Process Explorer from SysInternals) on your own or from your colleague. If you can also share how you use it then it would be fantastic.

1. My brain – always on
2. My hands – always impatient to get on to the next test
3. My feet – always on the move to the programmer with my latest bug findings
4. My mouth – to provide timely and critical information about the project, and help others with their testing struggles
5. The Needle Eye Pattern – to help decouple my test automation approach from the actual product, so that I can unit test my own code
6. Google – when I am out test ideas
7. Skype – when I am desperate about my testing strategy, I reach out to others who help me jiggle through it
8. Yammer – for communicating with colleagues how they solved a problem that seems like mine
9. telephone – to reach out for more direct feedback from some of my peers
10. My running shoes – when things get really stuck, a good one hour run helps me overcome my thinking impediments

Let us say there is a commenting feature for the blog post; there are 100 comments currently. How would you load / render every comment. Is it one by one or all 100 at once? Justify.

I would asynchronously pre-fetch some of the data, and load – say – 20 comments in the first batch. After that I would dynamically add the next comments in chunks. There could be a problem with threads while loading, but I will deal with the problem, when we encounter it.

Have you ever done check automation using open-source tools? How did you identify the checks and what value did you add by automating them? Explain.

I used FitNesse, Cucumber, Cuke4Duke, Robot Framework, JBehave, JUnit, Concordion. I identified the checks before developing the story. By the discussion I had with the programmers and the ProductOwners we transformed our absent tacit knowledge about the product and the domain towards a more ubiquitous tacit knowledge, at times we even gained specialist tacit knowledge in our domain in the long run. See Tacit and Explicit Knowledge from Harry Collins on a more in-depth discussion about the terms.

At times, I also had to intensify the already existing examples. I used my background in software testing with decision tables, domain testing, and the like to come up with new risks and new test – sorry – check ideas. By then I extended our check base.

What kind of information do you gather before starting to test a software? (Example: Purpose of this application)

I gather my mission together with my stakeholder. I try to find out what sort of information will be interesting. I will also consult with the developer about potential risks. Then I usually start a brief exploration of the application, only to get some overview. After that I will have identified in less than half a day what I should spent my time testing on, and what problems I already encountered. And yet, I use session-based exploration for this.

How do you achieve data coverage (Inputs coverage) for a specific form with text fields like mobile number, date of birth etc? There are so many character sets and how do you achieve the coverage? You could share your past experience. If not any then you can talk about how it could be done.

I identify the domain rules about that input field. Then I come up with a set of positive test cases. After that I try to break that behavior, and see what happens. That worked reasonable well in the ParkCalc challenge a while ago. After I saw that I could enter scientific numbers into the date and time fields, I could generate parking costs in the range of 5 billion or so. After that I might also be tempted to try things like sql-injection, and so on. When I run out of test ideas, I take a final on the test heuristics cheat sheet from Elisabeth Hendrickson, and see if I left anything crucial out of my approach.