Today I decided to join it: Twitter. After spending some time playing around with some settings and searching for some interesting people to follow, I am still wondering what makes people curious about it. What made me join then? Today I got a notice on Xing about the XP Days Germany on Twitter. Curiosity killed the cat. Personally I decided to try it out for some time and see what might happen. I included my updates on the right, as you might have noticed.

Since I swtiched to WordPress a few weeks ago, I observed comments to my entries. I realized that there are none. James Bach pointed out a misconfiguration to the comments requiring everyone to login before being to comment on entries. Unaware oft his issue I changed the settings this morning, hoping for more comments to come. Thank you, James.

Today I switched to WordPress and happily migrated all entries from the previous blog to this one. Please mind to update your rss-feed in case you subscribed to my old feed.

Lisa Crispin motivated this morning an entry on What I’d go back and tell myself. Compared to Lisa I would like to get back a few years more towards my time in university that started in 1999 (yes, I’m not that old zealot). At the time around 2001 I took a course called Software practical that was divided in a theoretical leacture and a practical group work to build a system. During the lecture there was eXtreme Programming mentioned as a new technique with a delightning practice called Test-driven development. By that time I did not get the point and it took until 2008 to realize the benefits of test-driven development. Going back to 2001 or even 1999 I would tell myself that I really should read about test-driven development, refactoring, design patterns and unit tests and dive into more topics. Nearly two years from now I started to read on these things and get the ideas from the great books out there. Currently I feel the need to get even with the books, but am really overwhelmed so far from the pile of books that lies in front of me.

Another thing I would like to tell myself going back into the past, is that I will truly make a good start in software testing and development and see the opportunities to improve my work. Going back to 2005 I would tell myself, that I should not waste the time to investigate on PhD positions at the universities, but rather get some real hands-on in the business. Most of practices and techniques I use today I learned when I was out from university and I’m glad to have learned all this stuff.

While reading the excellent xUnit Test Patterns book I was introduced to the idea that making something less private on a class in order to get a test for it in place may break encapsulation for good. While this pattern ensures that you just have a single instance of a configuration or memory consuming object in your production system, you will have a problem with unit testing it. The xUnit book provides a pattern to make this easy. However, the Substituted Singleton pattern relies on the fact that you don’t define the singleton instance on your original class as private, but rather as protected, so you can override the instance by a subclass where you have influencing methods so you can influence your test behaviour directly. This breaks the encapsulation of the class you wrote for production use, but for good, since you can now test all your users of that singleton class rather than needing to do some special treatments of your environment in order to achieve the same. A similar reasoning can be applied for not using the final modifier on classes. These ones you will not be able to test by then, too. When deciding to do just this, you’re trading off the ability to unit test your code and show that you produced a good quality while doing so and communicating your intents for the next programmer having to deal with your class by breaking encapsulation in your class hierarchy. The first overweighs the second here.

Currently I am faced with a similar situation at work in a different context. We use Perforce for version control at work. This tool is able to indirectly track renamed files not directly – you have to do an integrate to produce a copy with a new name and delete the old file afterwards for that. The current eclipse plugin for Perforce does not cope with this. It just deletes the old file after having created the new file and opened that one for add. By submitting this change you loose the version history and cannot see anymore that the previous name of a class was Foo and is now Bar or that initially you had put up the class in the package foo.bar and now is placed in foo.baz. One of my colleagues realized this and now mandates that we should do a semi-manual refactoring for these kind of operations (rename class, move class). Personally for me the same trade-off as for encapsulation of fields in singletons exists here. Having tool support for doing these refactorings completely automatically is a big win and I can live with the fact that I will not obviously see where the file was coming from in the version history. The class was renamed to hopefully some good reason – i.e. to communicate intent in a better way or to have it placed under a package, where it is reasonable. On a side note all the history I would need should be placed in the unit tests for the classes. This trade-off is fine for me and I don’t see good reasons to get into the semi-automated process of manually integrating a file in the version control system, just to see the complete history of the file. It’s fine if I just see, “Oh I added that file and deleted that one in the first change to that file, seems like a renaming operation.” and follow-up on that file if needed (I don’t read the revision history often – most of the time I don’t have to.)

While the values might just seem to be a list of higher level goals with no direct outcome on the particular project, there is a lower level of definition in Agile methodologies: Principles. From Elisabeth Hendrickson and Lisa Crispin & Janet Gregory I was able to construct a list of useful principles. Here is the first half of this list of testing principles.

• Provide Continuous Feedback

Feedback is one of the higher values for sucessful projects. As Poppendieck & Poppendieck pointed out in Lean Software Development – An Agile Toolkit feedback is vital to software development. They compare software development and manufacturing with cooking: While development or engineering means to come up with a recipe, manufacturing is just applying over and over the recipe that is already there. Since software development is a recipe building action, it needs iterations and feedback to be successful.

From this perspective testers provide feedback from the customer point-of-view into the programming activities. Likewise they provide feedback as a substitute for the end-consumer by adding exercising tests – may these be manual or automated. Like the memory structure in the current computer systems, testers can be thought of as the cache towards production for each program increment. This does not mean, that testers are the gatekeepers for any software deliveral. This rather means, that they can provide timely feedback to the programmers, rather than opening or closing the gate towards production. By addressing issues before the software gets into production, the feedback for the programming team is achieved in an efficient manner for them to learn.

• Deliver Value to the Customer

Testers are the bridge between the programming team with its technical terms and the customers with their business language. By incorporating a solid ubiquitous language into the test cases, these two worlds can get closed and guarantee transparency for the customer. This higher level goal can be achieved by working directly with the customer on test cases and likewise giving feedback to the team members who are developing the software the customer ordered. A test which is twisted up with complex logic is no value for the customer. A test case turning business terms green when it’s passing on the other hand is. Testers provide the service of filling the communication gap between technical software development and the business of the end user.

• Enable Face-to-Face Communication

Alistair Cockburn points out the principle, that two people at a whiteboard are communicating more efficient than two people over email or videotape. He visualizes this principle in a well understandable manner. When considering software testing, a tester should therefore motivate and enable direct communication forms wherever possible. When two people are talking face-to-face on a difficult requirement, they efficiently get a common understand of it. Therefore tester should enable direct communication to customers, to developers, to department heads, to project management, …

Lisa Crispin and Janet Gregory introduce the Law Of Three in their book on Agile Testing. The Law Of Three states, that any design decision or requirement discussion need to involve one person from all of the three groups: customers, developers and testers. Whenever a discussion is started, a decision need to involve all affected groups. Direct face-to-face communication supports this law by building up efficient communication channels between the three.

• Keep it Simple

Simplicity is a key in Agile Software Development. Without the waste of usual software processes teams maintain their ability for agility. This directly means to keep your test cases as clear as possible. Additionally through simplicity you enable your team to understand your tests and be able to adapt it if the business demand this. Just like the Collective Code-Ownership practice in eXtreme Programming, testing teams have the demand for collective test case ownership. Just through simplicity you not only enable your co-testers, but also the business analysts, developers and customers to adapt test cases when the situation asks for this.

• Practice Continuous Improvement

This principle is simply demanded by the craft aspect behind Software Development. Just like any new cool programming language or technique, testers should keep their improvement potential up-to-date. By visiting conferences on test techniques or latest test automation tools, a tester can improve her skills in these areas. This also means, that learning a new programming language or test technique such as performance- or load-testing is a way to improve one’s own abilities. Learning a common language to express testing patterns is another example of it. By providing this pattern language to your teammates, you enable them to learn from you and additionally you improve your speaking skills. These are just some parts of your daily work where you can practice improvement on yourself and your team. Look out for additional possibilities.

• Reduce test documentation overhead

As Mary and Tom Poppendieck pointed out in Lean Software Development – An Agile Toolkit wasteful tasks should be avoided in order to have the highest possible return on investment. This means reducing test plans, test design documents and test protocols using tool support and ubiquitous language to a minimum. This does not mean to drop any test documentation. Rather your team and your customer should be brought in to the discussion of the level of necessary documentation. In case you’re working in safety critical software or your organisation demands it due to CMMI or ISO9000 compliance, you will be forced to use more extensive test documentation. Look out for ways to reduce these to a minimum and avoid wasteful documentation, which needs adaptation to the current situation every week or even on a daily basis.

Currently there is a lot of discussion ongoing on the causes of failing agile teams. After watching these discussions for quite a while now, here is my view of the topic. The brief quint-essence is: If you’re doing crap, it doesn’t care whether you call it Agile or not – you’re just doing crap. If you’re doing a good job, it doesn’t care whether you call it Agile or not either – you’re just doing a good job. About a year ago I started to take a look on how Agile Software Development is done, especially under the circumstances of testing on Agile Teams. Since I realized that my company was struggling with delivery based on waterfall projects and non-cross-functional teams, Agile seemed to be a solution worthwhile to take a look on. Since introducing change in working habits is a large effort and I was not in a key position to introduce this, I started especially focused on the mind-set behind Agile Software Development and the sucess aspects of it. Incrementally I achieved to introduce changes based on the practices from XP – but for several reasons I was confident, that I could not introduce all aspects of it. Today I would say that I did not take the courage to just change stuff and come up with the success story afterwards.

The changes I started to introduce in my company where tiny. We were suffering from shell-scripted test automation, that was very interface-sensitive. A change in one of our software components could lead to a full regression test result disaster, our testing team being unable to give the stamp of approval on it safely. Manual testing was no practical option, since it would have lead to many weeks of efforts for that, but the bugfixes we had on our desk were critical for our customer. My team decided to reimplement test automation with a new approach and we decided to use FIT for it.

The next six weeks were spent focussing on the highest priority business use cases. We were able to build a new framework for testing on it. While I was reading The Art of Agile Development from James Shore and Shane Warden, I brought in many aspects of the practices and underlying principles and values into my team. Though we were not implementing the full set of practices, we were quite successful over the next few months and finally finished our energized efforts with a test framework, so that we were able to run our group during vacational time with just 1.5 persons, while we were unable to hold the pace one year before with 10 persons.

In the meantime one of my colleagues as well introduced a testing framework based on FIT but with a different approach. While my team used Java, his used Python. While my team focused on the readability of the tests from a business perspective reducing all visible test data to the necessary level of detail, his team put every little detail for the tests in there – whether they contributed to the expected results or not. From organizational point of view it was decided, that we needed to focus on just one programming for easier job advertisements. We came up with the solution to use Java as programming language, since most of our developers have either a Java background or are Java Developers. We started to move the previous Python classes to Java, without having the people introduced to Java beforehand. I did not that hard participate in the migration of the classes and see today that there is bunch of technical debt introduced. From my point of view there are two flaws that result from neglecting the craftsman aspect of our work: the table structure and the Java class code.

While I had read the book on FIT before starting to do anything, my colleague just started. Since he did not get the recommendations from people who have invented the framework and the problems they see on several topics, the tables are interconnected and while reading them I get a brain-twister. The second problem was, that we were forced to stick to the wrong table layout while getting the fixture code from Python to Java. As well since the people doing the work were new to Java, they did not come up with Pair Programming, Unit-Tests for the fixture code and no reusable design in the mind. Getting the toolset now ready for the next customer, will be an immense amount of work, if we agree to stick to the table layout, from my point of view.

The conclusion from that lesson for me is, that Robert C. Martin is of course right. With proper introduction to Java, JUnit and FIT the solution would be better today. We did not say, we were implementing Agile. We just focused on the parts of Agile, that helped us very much. The first approach from my side was a success, because we were simply doing a good job with future opportunities in mind and did just the necessary stuff skipping most of the YAGNI implementations. We were not doing Agile, we were not calling it Agile, we were just doing a good job. On the second step, we were not doing Agile, either. We were not calling it Agile, either. We were just doing a bad job, not paying attention to the skill-set of the people doing the implementation, not looking over the code before or even directly after submitting, no Pair Programming. We were doing a bad job, that’s it.

As stated intially, it does not matter, whether you do a good job and call it Agile or not, or whether you do crap and call it Agile or not. In either case you either succeed or fail, no matter what name you give that child.

Further reading:

• From Agile Development to the New Software Engineering (Column by Alistair Cockburn)

• How to Fail with Agile (article by Clinton Keith and Mike Cohn)

• Why can’t people deliver? (blog entry by Alistair Cockburn)

• Quintessence: The fifth element for the Agile Manifesto (blog entry by Robert C. Martin)

• Testers: The Hidden Resource (blog entry by Lisa Crispin)

• The Decline and Fall of Agile (blog entry by James Shore)

• Dirty Rotten ScrumDrels (blog entry by Robert C. Martin)

Lately I exchanged some thought with Alistair Cockburn on his new side over e-mail. Therein I introduced him to some thoughts that raised in my mind while reading through Agile Software Development – The Cooperative Game and Extreme Programming Explained and he told me to try the terms out for a while. Therefore I decided to introduce the thinking behind it for everyone. To introduce you in the topic I will first explore some definitions from the Extreme Programming (XP) and Agile world and try to relate them to the thoughts from the Cooperative Game.

Extreme Programming

This chapter describes the initial thoughts from Kent Beck on the relationship between practices, principles and values in XP. The initial of view of Kent are complemented with the thoughts from James Shore.

Practices

The XP practices build up a key set of actions to follow in order to get into the XP way of thinking. Practices are the entry point for teams new to XP. They build the first insights on how work is being done in the XP way of thinking. Practices are clear, so that everyone knows if they are followed. Practices are situated in that they just tell you what to do in a particular circumstance. They are not applyable in the rest of your life. In the second edition of Extreme Programming Explained Kent distinguishes between two levels of practices: the Primary and the Corollary.

The Primary Practices are introduced to get started right from scratch. These build up practices everyone should be able to start with right now without any dependence – beside the will to apply them. Among these practices there are advises like Sit Together, Whole Team, Pair Programming, Slack, Ten-Minute Build and Continuous Integration – just to name a few. The Corollary ones build up practices which might be difficult to implement without following the Primary ones. Among these there are Team Continuity, Code and Tests, Single Code Base or Daily Deployment.

Principles

As Mr. Beck describes in a wonderful picture, since practices and values are an ocean apart, principles build the bridge between them. The principles make the clear practices relate to the universal values. Principles make the XP team realise, why they are doing a particular practice and lead to the higher-level, universal values behind these. Among the XP principles there are Humanity, Mutual Benefit, Reflection, Flow, Quality or Baby Steps – just to name some examples.

Values

Values form the universal underlying part of the practices. XP values Communication, Simplicity, Feedback, Courage and Respect. The underlying value of communication, courage, respect and feedback highly motivate to apply sitting the whole team together as near as possible for example. The principle of humanity demands this. Without the values behind the practices, the latter would be tedious activities fulfilled just since it is denoted somewhere. Values form the motivating factor of the XP practices, while principles give reasons to apply them.

Agile

As Robert C. Martin wrote on Object Mentor summarizes how XP relates to the four Agile values and the twelve principles mentioned in the Agile Manifesto. Alistair Cockburn writes, that the people who signed the Agile Manifesto did not dare to go deeper into the materia and try to agree on common practices. Robert C. Martin perfectly describes, why the XP practices meet the Agile values and principles, but XP is just one of several applications of the Agile values and principles turned into practices.

The Cooperative Game

Alistair Cockburn introduces in the Cooperative Game the ShuHaRi levels of skillset development. The small words Shu, Ha and Ri are taken from the Aikido domain and Alistair maps them to the skillset in software development.

Shu

Shu means to keep, protect or maintain.

In the Aikido domain the Shu level is related to a student just beginning to learn the techniques. Therefore the Shu level is the entry point for new skills to learn. The student is taught to apply the techniques by copying them and follow the teacher’s advises by his words. The protection on the Shu level is to avoid early distraction of the student by the several existing techniques around. The student shall be able to focus on practical advises – on practices.

Ha

Ha means to detach and means that the student breaks free from the traditions.

After following the practices, the student starts to question the practices rather than doing just repetetive activities. At this second level it is realised what the underlying purpose of the initial teachings is. In the human grow-up process this stage could be compared to the puberty of teenagers, who start to question the practices they initially just copied from their parents and other children in their surrounding environment.

Ri

Ri means to go beyond or transcend.

On the third and last stage the student adapts the previously learned practices to meet his own style. While applying the principles she just became aware of, the underlying values are realised and followed.

Conclusion

Basically my current picture of Agile, XP and ShuHaRi consists of practices to follow on the Shu entry level of skillset development. When trying to get an Agile mindset, you need to start with particular practices on the Shu level. Since you just start to get into it, you need to learn the practices by the word in order to not get distracted by your previous working-habits and values.

On the Ha level there are principles which give you the reasons for particular practices. When the applications of the underlying practices got firm into your mindset, you will begin to question particular activities. You will only get to realise the underlying principles, if you have followed the practices on the Shu level long and thorough enough. This is the point in time when adaptation has taken roots. You should be now in place to see the larger picture behind your working-habits and be able to begin to change some of the details and try to fit variations of the practices to your particular circumstances. You should be able to start adaptation. For this phase to start it is essential to have followed the practices by the book – as strinkingly denoted in How To Fail With Agile.

After realizing what does work and what not through adaptation, you will be able to see the universal parts of your new skills. On the Ri level you will start implementing the values naturally without even thinking about them.

To conclude my proposal, the Ha level of skillset development builds a bridge between the Shu and the Ri level. On the Shu level you follow the practices by the book, on the Ha level you start to realise the underlying principles of the particular methodology you’re following and on the Ri level you see the whole world in values and start adapting your own style. This is the quintessence.

Outlook

Since I have a strong background in software testing, personally I would like to try to define practices on the Shu level, principles on the Ha level and values on the Ri level for software testers. Elisabeth Hendrickson, Lisa Crispin and Brian Marick did very good thoughts on the theses topics, but I have not seen the ideas structured as I tried to sketch here. If my proposal turns out to be valueable, this might become future work for Agilists.

Since I asked Alistair Cockburn to do a review of the text, he already was able to add a note on his blog on it. Thanks Alistair.

After slipping through some of the blog entries from Alistair Cockburn, I found the following entry: Everyone should be a methodologist. After slipping through the errors of early methodologist in part 2 of that entry, I realised, that I personally ran into point 2. and 3. lately. Likewise I see many times points 4 and 5 forced in my company. Thanks Alistair for reminding this to me. If you got interested by the content, I recommend reading Alistair’s book Agile Software Development – The Cooperative Game, which covers many, many more aspects.