Robert C. Martin brought up a blog entry yesterday called The Polyglot Tester. He argues about the Behavior-driven development style of automated tests the table style encouraged from FIT, Slim, RobotFramework and others. While I’m not going to jump into the discussion which of the existing tools is better, worse or more suitable to do some job, here are some references to patterns for test automation, which I came across.
Given/When/Then
The given-when-then (GWT) style of expressing an automated test seems to be origined in Behavior-driven Development. The mechanics are simple. Let me repeat the example Uncle Bob used in his blog entry:
Given that I am a user named Bob with password xyzzy
When I log in with username Bob and password xyzzy
Then I should see “Welcome Bob, you have logged in”.
This test describes a login mechanism. The given statement makes all preconditions of the test clear. In the example above the role of the test is expressed explicitly. In addition the technical pre-conditions are also mentioned in the form that there exists an account with the name “Bob” and the password given. There may be additional pre-conditions that our test might need. I may describe them with some “and” clause like
and the system is running
The when statement clarifies the actions the user – in this case the test driver – does. In the example we are referring to login into the system with the username and the password stated in the pre-conditions.
The then statement asserts all post-conditions that the system shall realize. Again, in the example above the test shall assert to view the welcome screen with the logged in user printed out.
Pros
The pros of this approach include speaking tests. In fact, business stakeholders can read the tests and understand what the business rules behind the tests are verifying. They might even be able to write some of these tests on their own.
Cons
While the mechanics to express such an automated test in this manner should be easy to follow, Robert Martin expresses that the approach uses a lot of words to describe what should be happening. In general in order to understand what a test does you have to read a lot if it’s written down in a confusing manner.
Four-Phase Test
The Four-Phase Test or Build Operate Check pattern is a generalization of the Given/When/Then style. Therefore the structure is similar. The four phases include:
- SetUp or Build
- Execute or Operate
- Verify or Check
- and an optional TearDown.
Comparing the approach to the Given/When/Then style, there are parallels. The Given is the SetUp or Build part of the Four-Phase Test, the When part is the Execute or Operate part and finally the Then part is the Verify of Check part. The difference between the two is the fact that the Four-Phase Test pattern may not make these phases explicit. For example consider the Four-Phase Test Robert Martin uses in his blog entry:
| Deposit | Credits |
|---|---|
| 25 | 1 |
| 1.00 | 5 |
| 5.00 | 30 |
While it may not be obvious that there are four phases in the test data mentioned above, they are there, but expressed in a very efficient manner leaving out unnecessary complexity. The pre-conditions are to have an implicit deposit of 0, the execution is mentioned in the deposit column, the post-conditions are mentioned in the credits column, the teardown is either left out or not necessary.
Pros
I will avoid mentioning the pros and cons from the behavior-driven development GWT style. They apply to some degree also for the four-phase tests. The special advantage of the table style is it’s conciseness. It’s very easy to get an overview of the underlying business rules.
Cons
On the contra side you need to know some of the left out complexity. For example the fact that the deposit value needs to be zero is not mentioned explicitly any longer. A good (exploratory) tester would ask what would happen when I got three quarters deposited and put in another one. What happens at 4.75? Of course with different pre-conditions the mentioned post-conditions would no longer hold. Maybe the machine has some person detection built in. Does it matter if I put the coins in or my colleague? These details of the system are left out for this test to focus on the business value behind it and you need an understanding of the underlying assumptions to be able to read the test. At least business stakeholders may need an introduction if these are written badly.
Flow style
For some reasons I haven’t found out yet why we use the flow style of writing down automated tests a lot a work, but we do. In this style you may write down what you need to have, what you need to exercise and what you want to verify. As you might have figured I combine the flow style a lot with the Four-phase test approach. In our system we describe what kind of account we need for the pre-conditions, describing what products the account needs to have sold, how much money the account shall have. Then we exercise some service in our system under test. We describe the parameters for the service in the execute part. Then we check all post-conditions, like the lifecycles of the products initially sold and the money the account has left or got available in addition to the pre-conditions. On a coarser level we maintained the Four-phase test approach, but used the flow style to express what happens from the business position.
Pros
The flow style enables you to write down anything you may want to focus on. You are flexible on what to express and you can combine the flow style with any other techniquie you know.
Cons
With a greater degree of freedom, there comes the risk that you write down your business rules in a manner that leads to high-test maintenance costs. You may write down your automated tests in a way that you will have to change every single test in order to adapt to changes in the business rules. In general, principles of good software development also apply to your automated tests. Foremost this includes the DRY (don’t repeat yourself) principle based on copying & pasting.
Summary
Overall I showed the three styles of automated tests that I’ve used in the past year. I am pretty sure there are more ways to express tests for the purpose of automating them. However, I find these useful in many ways. The Given/When/Then style proved particular good when dealing with a simpler application in our domain, that did some XML post-processing. Describing the pre-conditions and generating the input XML from it, was very easy. For the check of the post-conditions I relied on assertions based on XPath expressions in the produced output. Most of the time I make use of the Four-phase test pattern for my tests. Mostly it is combined with some flow style based test approach like I mentioned above.
In addition I learned how to do xUnit based testing. Overall I try to learn techniques and incorporate them into my daily work as good as possible. The three mentioned approaches together with xUnit style tests form tools in my toolbelt which I may choose or not choose based on the circumstances in which I want to apply them. By learning new approaches, I’m able to select wisely from these and make conscious decisions about their applicability. Therefore, the more approaches I know and the more approaches I am familiar with, the better.
