On February 2nd 2011 I was invited by Gojko Adzic to a seminar on Winning Big with Specification by Example. It was a half day seminar that took place in London. This is what I took away from the course.

Gojko used regular group reflections probably motivated by the Accelerated Learning method, which Sharon Bowman covers in her book Training from the back of the room. This made the learning outcome and exchange among participants very worthwhile for me, since I got some insights on problems that other companies faced. In my reflection group there was a lady which stated she was a business expert (I don’t recall whether she was a Product Owner or part of the Product Management team unfortunately), and was eager to learn more about this stuff.

Motivation

In order to warm-up we exchanged our thoughts on Specification by Example, and what we already did at the companies we work for. I shared some of the stories and lessons I learned since I started for it-agile GmbH in September, and the different teams I worked with in order to get them started with Agile Acceptance Testing and Specification by Example. From the others in my group I gained the insights that they were stuck with some of the stuff, tried already some things out, and didn’t seem to succeed with it.

Gojko’s first slide deck covered the key benefits of Specification by Example. He interviewed more than 50 teams for his upcoming book Specification by Example, and stated some citations from them. From the interview material, he had extracted four key benefits: higher product quality, improved collaboration between testers and programmers (which he calls better tester/developer alignment), implementing changes more efficiently, and less rework. He cited from his interviews that the teams were able to build the right product right from the start. Since the customer perceived a higher quality by using Specification by Example, the teams were convinced of the method. When you get the software right the first time, you end up with less rework, of course. Since Specification by Example relies on a shared understanding of the thing to be built, the teams worked closer together. Finally, when the teams found out how to make changes to the system, and which other areas might be affected by these changes, they set themselves up for more efficient changes to the software.

The process

After the next discussion round, Gojko introduced us to the key steps in the process. The next two days after the seminar, I attended a course on Visual Facilitation, and worked together with two colleagues – Nico Botzet and Sebastian Sanitz – to incorporate the flow on a chart. Though the texts are German, you should be able to follow along.

Gojko explained that at the beginning of the flow are business goals. He explicitly warned to start with user stories or use cases directly. Every team he interviewed with that started with user stories or use cases ended up to abandon this, since it didn’t work for them. He continued that the first step is to derive the scope from the business goal. I’m still a bit unsure about this step, and I surely have trouble understanding how this is done. When describing this point in my courses, I always have trouble diving into it. I think teams are best set up to start with everything they have up to that point, and get started with the later steps first during the adoption, and to find out what works for them based upon business goals, use cases, or user stories later.

After the scope for the feature to be implemented has been identified, the team gets together in some way or another in order to describe using examples what the underlying feature is going to look like when it’s implemented. Gojko explained that specification workshops may work for one team, while some teams use informal meetings instead. After this step, from the derived scope there are some examples which describe the functionality.

Gojko mentioned that he hadn’t incorporated this step in his book Bridging the Communication Gap, but while researching his latest book, he found out that these initial examples need to be refined. So, usually a tester sits down with a business expert and works through corner conditions of the functionality at some point. Thereby the examples get refined, yielding a Specification by Examples.

While the software is being implemented, this specification is automated without changing the wording – most of the time even using the initial set of specification directly. There are different tools around which support the automation process. These are Agile-friendly frameworks like FitNesse, Cucumber, Robot Framework, or commercial ones like Twist from Thoughtworks. Gojko said, that Specification by Example is not reliant on a tool. Indeed most teams who started to implement a tool – just got that, and implementation of a tool, and not a working approach.

Automating the specification by example yields an executable specification. Instead of heavy documents, which mostly are not worth the paper they are printed on, the executable specification shows directly whether the software works according to it or not. This also has the advantage of a specification which is nearly never outdated – and if so, you are just one tip on your keyboard away to get you the feedback whether your specification is up-to-date or not.

When running these specifications frequently – i.e. in following iterations – the system turns into a living documentation. This documentation can be consulted to check for impacts on upcoming changes, to check for the behavior of your software, or to help new colleagues get introduced to your software system. Thereby the whole team has an advantage on this – and some of the teams Gojko interviewed with even had a competitve advantage after having implemented the approach.

The remaining afternoon Gojko spend on telling success stories using Specification by Example. He got into Specification Workshops, introduced us to some adoption strategies, and showed how it fits into Agile software development. Most of this I already had read in Bridging the Communication Gap and while reviewing Specification by Example, so I won’t repeat it here, but encourage you to either buy on of the two books, or visit one of Gojko’s courses.

Conclusion

One particular thing that I had realized while Gojko was telling the stories from the teams he interviewed, is that the teams start with one implementation of the approach, and start to reflect on their course regularly and adapt. He told several stories, where the teams started with a basic understanding of ATDD or Specification by Example, and later found out that something was not working for them. All of the teams reflected over their course about half a year later, and found some necessary adaptations. After changing their course, they continued for a while, until they eventually found the next problem. Basically this is an instance of Rudy’s Rutabaga Rule (from Weinberg’s The Secrets of Consulting):

Once you eliminate your number one problem, number two gets a promotion.

(page 15)

Once the teams solved their (perceived) number one problem, they found out, that there was a number two problem lurking – which they then also fixed after some time. This now makes me wonder whether the approach can be successfully implemented in a more traditional company or project, where reflection over the problem might take longer. At least I am convinced that it will take traditional companies longer to succeed with Specification by Example – on average.

Another key take-away for me is a two step process for adopting Specification by Example. In the examples Gojko referenced to there was a pre-iteration phase where some team members discussed the examples, and a second phase, where the examples got refined. Some teams used a specification workshop for this, other teams succeeded by having two or three people working on the examples, yet another team had informal meetings with everyone interested in the function attending. When the iteration started, these initial examples were refined by either some team members, all team members, or anyone that had an interest in it. I think this is an insight that Gojko was also referring to, which he didn’t see while writing up Bridging the Communication Gap. This refinement step seems to be necessary, and different teams found different clever ways to succeed with it in their context. Personally I hope that the AA-FTT pattern writing community will investigate this pattern in more depth over the course of the next few years – and I hope that I can find the time to contribute to it.