The other day, I sat down Kishen Simbhoedatpanday in order to talk about ATDD, and eventually an upcoming class on the implementation side of ATDD. We talked about maintainable tests, and how you could refactor tests to yield better tests. Gojko wrote about the anatomy of a good acceptance test a while back, and I think we can be more explicit here. Then it struck me. The Single Responsibility Principle also applies to your automated examples – and why that’s the case. You just need to think on a conceptual level about it. Mixing business domain concerns with application concerns in your examples – and sometimes even with driver concerns (think Selenium) – is a terrible thing to do.

Let’s explore this idea further.

The problem

An example would be handy right now. So let’s take a look. Here is an example that I found with a quick google search:

Feature: As a user I should be able to perform a simple google search

Scenario: A simple google search scenario
Given I am on the main google search
When I fill in “q” with “ATDD by example”
And I click “gbqfb” button
And I click on the first result
Then I should see “ATDD by Example: A Practical Guide to Acceptance Test-Driven Development (Addison-Wesley Signature Series (Beck))”

What’s wrong here? A while back I thought the problem lies in several levels of abstractions in the same textual description. But now, I would like to stress the point that there are different concepts dealt with.

For example, the line When I fill in “q” with “ATDD by example” deals with the domain concept of the search term together with the application domain of the html-page, and how to drive the browser using a driver library. Similarly, the line And I click “gbqfb” button deals with the application domain, and the business domain. The business domain is “searching”, the application domain is the particular implementation detail that there is a button for searching.

The problem lies in the different concepts involved here. There are three problems with these:

• They are harder to write
• They are harder to read
• They are harder to maintain

Harder to write

Wait, what? They reflect how the application is implemented, so they are quite easy to write, right? Wrong.

To be frank, I have written such tests a lot in my life. From those dark days, I remember a couple of things. What I remember mostly is that I got home pretty tired. I needed to put in so many hours to remember all the various contexts I dealt with. “If I exercise the application this way, and then see this happening in the domain, then there is a problem.”

Can you spot it? Yeah, it’s context switching at its best. So, writing these kind of tests is a whole lot harder rather than grasping the business domain, and clearly expressing that. You need to keep track about various things, on the business domain side, and the application domain side, and sometimes even on the technical driver domain side.

Now, compare the above piece with this example:

Feature: As a user I should be able to perform a simple google search

Scenario: A simple google search scenario
When I search for “ATDD by example”
Then I should see one result from “amazon.com”

Is that clearer? Maybe. How hard was it to write that thing? Not that hard. I didn’t need to remember the tiny nifty details about the html page, where the search field is, and which button to press. I focus the test on the business domain: “search for something, and expect at least one result from that side”. Straight-forward for me.

But, but, but, you can record these tests easily! That’s right. But think again: what happens when that test fails in five months?

Harder to read

So, basically the same argument applies as well for the reading part of the tests. You need to do a lot of context-switching for the involved expressed domains. In the example above I counted at least three domains in five lines. And that’s merely one scenario, one test. How large is your test suite?

Clearly, unfocused tests can become a great deal of pain when trying to read them and find out what the problem is. Tests that deal with several domains at the same time are unfocused. You need to keep track of several domain concepts. You need to translate between them, in order to keep track. It’s like you read three stories at the same time: the story from the business, the story from the application, and the story from the test automation.

Compare that with the second example I provided. Is that clearer? I think so. Since it’s focused on the business domain, we can focus on reading that piece. That also explains why I like to write unit tests for my test automation code. Because I need to deal with all the fluff that is in place there to express how I hooked up the application to my test automation code. I want to make sure to have tests for that domain of my problem as well. Oh, and of course, you also need application unit tests in the end. We certainly shouldn’t forget these.

Harder to maintain

But the biggest problem lies in the long-term maintainability. Think about it. If your examples express concerns from three different domains, your tests are highly coupled to these three domains. So, when you change the driver for your application, your tests might go bye-bye. If your application changes, your tests might go bye-bye. If there is a change in business rules, your tests might go bye-bye. Sounds like a lot of maintenance risk to me.

So, the way to go here, is to separate the three different concerns, each into its own place in the whole code base. The business domain should go with the examples. The application domain should go with the system under test, or if you need to translate between the driver for your tests and your application, then you want to address that in your support code. Oh, and you want to test your stuff that is using a particular test driver as well.

Three different concerns, three different ways to tell you that something had changed there.

What to do about it?

Separation of concerns is not a new concept in software development. The hard part is to spot when we mix concerns, and then separate these concerns accordingly again. In the end, it will result in fewer work writing these tests, fewer work reading these tests, and fewer work maintaining these tests. So, it sounds worthwhile to follow that tiny extra effort.

Can you spot more problems in the code above? Can you spot some Single Responsibility Principle violations in your acceptance tests? We need to develop a sense for finding these kind of smells.