The other day I was going through the first few drafts of the German translation of ATDD by Example – A practical guide to Acceptance Test-driven Development. Since some days I had been watching the list of issues that Dan Woodward and Arjan Molenaar were heavily working on some updated to the UI system, and FitNesse in general. Then Mike Scott told me that he just did a presentation at the Agile Testing Days, and used the recent version of FitNesse from the CI-server – and it looked awesome. Putting all together I decided to use new screenshots from FitNesse in the German translation. Until I get to update the English version of the book, here are some screenshots for the next version, which should be officially announced within the next week or so.
Continue reading What’s next in FitNesse?Category Archives: FitNesse
Testing Tools Mash-Ups
One of the three things that bring innovations, is copulation. That’s basically two already good ideas having sex with each other. I learned this from Jerry Weinberg’s Becoming a technical leader. The longer I stay in test automation and agile, though, I think this does not hold for combinations of testing tools, like a testing framework, and a UI-driver.
Continue reading Testing Tools Mash-UpsRESTful CI Integration with FitNesse
FitNesse offers RESTful command line services. In the past week I have worked on a solution to integrate these command line option in a Jenkins environment with the FitNesse plug-in. It turns out that this didn’t work as smooth as expected for me. I needed some special shell-fu to get the plugin and the most recent version of FitNesse working together. Here is how I did this, hoping that someone else will find my solution to help them in their aid.
Continue reading RESTful CI Integration with FitNesseParkCalc automation – Keyword-driven tests in FitNesse
To conclude the ParkCalc mini-series, I choose to work through test automation using keywords with FitNesse. As I was using Selenium mostly with RobotFramework, I decided to use Selenesse for the integration into the FitNesse environment. Here is the write-up as I implement the tests.
Continue reading ParkCalc automation – Keyword-driven tests in FitNesseFitNesse Search & Replace
Uncle Bob announced today the release of FitNesse 20100303. Please grab it from the FitNesse download page. Since I added a new function which I thought over the course of nearly the whole last year and which is not yet finished, let me try to introduce the idea I have with it.
The present
The feature I’m talking about is not very complex nor is it complicated. It’s a feature I was missing over the course of the last two years while working with FitNesse. It’s a simple regular expression based search & replace functionality. Well, thus far there has been a the possibility to do search & replace on the shell-level with regular tools like sed and others. So, what’s new about this? Well, at work we use perforce for revision control. Some while back I wrote a PerforceCmSystem for FitNesse to deal with opening files for writing, for add and for delete. Thereby we can store the FitNesse pages in our revision control system with real ease. Now, I get back to the need for a search & replace function inside FitNesse itself. On the shell we would need to check out each page to change, make the change, then check the changes back in. With search & replace I can now simply hit a button on a page after filling in stuff in two text fields, and then submit the changes made to the pages by FitNesse itself. Version control is done by FitNesse, as well as the change necessary. Great thing!
How does it work?
First of all consider the function to be experimental. There are no checks for consistency between the regular expression and the replacements. This made the implementation really easy, and there already some changes waiting for the next release to implement. That said, you will find two new text boxes on the refactoring page. In the first box you can enter regular expressions according to the description from the Java API documentation. You may use grouping enclosed with (), etc. On the second box you can insert a replacement, maybe referencing groups from the pattern in the first box. So, let’s walk through an example. Say, I have got some test pages, where Peter is used as the standard user. Now, for equality reasons, I would like to change Peter to Petra. Then I enter Peter in the first box, and Petra in the second one, hit the search & replace button, and watch the result popping up.
There is one thing you have to keep in mind. Search & replace works on the page hierarchy level. This means that any page below the page where the refactoring button was pressed, search & replace will work on. On any level above that nothing will be changed. So, you have to be aware where you start your replacements.
The future
There a two main streams for the future on FitNesse and refactorings. First, search & replace needs further evaluation and needs to grow. Uncle Bob suggested to include a two page flow, where I get all the results from the search on a second page and can choose among the replacements I would like to have – or as it is today, to replace all. This is already planned on pivotal tracker, and I would be happy to learn something new while implementing this.
Second thing I see coming is a refactoring for the tables inside FitNesse. With the underlying classes it’s very easy to write a custom refactoring for FitNesse that exchanges for example two columns from a table. For the following table:
| a | b | c |
|---|---|---|
| asdf | qwer | yxcv |
I might provide to exchange column a and c, resulting in the following table:
| c | b | a |
|---|---|---|
| yxcv | qwer | asdf |
So, I would be happy on feedback on this feature and the future of refactoring in tools like FitNesse. The column replacer would be a neat feature, but I hope it’s just the beginning of a refactoring tool for wiki-pages. So, please leave me a comment what you think about it, or how you think search & replace might be used as well.
Writing automated business-facing tests
Since I work in a more traditional orientated environment, I’m facing some questions regarding the usage of test frameworks such as FitNesse, Robot Framework or Concordion. The biggest problem I hear very often is to start implementing fixture code before any test data in terms of test tables or html pages is available. Directly the scene from J.B. Rainsberger’s Integration tests are a scam talk comes to mind where he slaps the bad programmer’s hand. Bad! Bad, bad, bad! Never, ever do this. So, here is a more elaborate explanation, which I hope I may use to reference to my colleagues.
So, the first thing is to pick up the classics on the topic and check the documentation about it. So, let’s start with the classic FIT for Developing Software. So, the structure of the book is separated into a first part mentioning the table layouts, in the second part goes into detail about the classes for the implementation. So, Ward Cunningham and Rick Mugridge seem to follow this pattern. Great. Next reference, Bridging the Communication Gap. Gojko introduces there specifications workshops and specification by example. Both are based on defining the test data first, and later on automate them. This helps building up the ubiquitous language on the project at hand.
But there is more to it. Since test automation is software development, let me pick an example from the world of software development. Over the years, Big design Up-front has become an anti-pattern in software development. Though there are some pros to it, on the con-side there are that I may try to think about each and every case which I might need for my test data, but I may be wrong about that. So, just in case you are not from Nostradamus’ family, thinking about your design too much up-front my lead to over-design. This is why Agile software development emphasizes emergent designs and the simplest thing that could possibly work. Say, if I work now on ten classes, which I completely do not need when the test data is noted down, then I have spent precious time on building these – probably even without executing them. When later on the need for twenty additional classes arises, the time spent on those initial useless ten classes cannot be recovered to cope up. Additionally these ten classes may now make my suffer from Technical Debt, since I need to maintain them – just in case I may need them later. Maybe the time spent initially on the ten useless classes would have been better spent on getting down the business cases properly in first place – for those who wonder why your pants are always on fire.
Last, if I retrofit my test data to the available functions in the code, I have to put unnecessary detail into my tests. The FIT book as well as the Concordion hints page lists this as a malpractice or smell. For example, if I need an account for my test case and I am retrofitting it to a full-blown function call which takes a comma-separated list of products to be associated with the account, a billing day, a comma-separated list of optional product features and a language identifier as parameters, I would write something like this:
| create account | myAccount | product1,product2,product3 | 5 | feature1,feature2,feature3 | EN |
When I can apply wishful thinking to my test data, I would be able to write it down as brief as possible. So, if I don’t need to care about the particular products and features sold, I may as well boil the above table down to this:
| create account | myAccount |
In addition to this simplification think about the implications a change for create account in the example above would have, when I need to a add a new parameter for example a the last billed amount for that account. If I came up with six-hundred test tables by the time of introduction of this additional feature, I would have to change all of those six-hundred tests. This time for changing these six-hundred tests will not be available to test the application. Wasted – by my own fault earlier!
In the end, it boils down to this little sentence I used to describe this blog entry briefly on twitter:
When writing automated business-facing tests, start with the test data (the what), not the fixture to automate it (the how). ALWAYS!
Adaptation and Improvisation
Dale Emery influenced this blog entry with his paper on Writing maintainable automated tests. At the essence I’m going to compare several test approaches: Unit Testing (i.e. Test-Driven Development), Functional or Acceptance Testing and Exploratory Testing. I’ll look at means necessary for regression capability (checking), costs for testing and costs for adaptation to changing requirements.
Continue reading Adaptation and ImprovisationPerforceCmSystem for FitNesse
Today I finally set up a project page for the PerforceCmSystem that I worked out initially for the 20090214 release of FitNesse. The implementation is simple and just enables your fitnesse pages for version with FitNesse by opening new pages for add, opening already submitted pages for edit, and deleting removed pages in the default changelist. Anyways you will still have to take care of the submission process by yourself when doing so, so don’t consider this to be the silver bullet. After updating to FitNesse 20090321 which was published yesterday by Robert C. Martin I decided to add downloads compiled for Java 1.5 and above to the project page. Let me know of any issues you’re facing.
Not every hammer fits all nails
I would like to refer to gojko’s latest blog entry, which made me a bit thoughtful. Here is the actual link I’m referencing: Using FitNesse pages as templates.
At my company we started back around easter to get rid of our old tests, which were
- build on a script generator, which generated shell-scripts
- had a “shared fixture” strategy with high interface sensitivity to nearly all DOC of the SUT
- suffered from test chaining and slow tests
I’m glad I now know all those fancy words after reading through xUnit Test Patterns :-)
We started using FitNesse. Actually I was the only one in our five person team, who had read the book. By coming up with mechanisms to be able to use “Shared Fixtures” and re-use an add-on component of our SUT for test account maintenance, we introduced a good framework, which is easily extensible etc. Since we did not get the support from our developers, which would have been necessary, we had to find ways to compensate for that by introducing unit tests for our generic test helpers, using refactoring and so on.
Parallel to our approach a colleague started to build a framework based on FitNesse as well. At the moment there are
- complicated long tables
- high level of details
- test table chaining
built into this test suite.
Currently I’m facing the situation, that we got the assignment to unify both approaches somehow. Since I’m pretty much convinced by the context-driven school of testing, I believe that both approaches have more or less a good reason to exist. After reading through gojko’s blogentry I think, that screwing up the system by not cooperating should not be a general solution. Since FitNesse has several ways to screw things up, stay aware of the hammer argument:
If I got a new hammer, everything looks like a nail.
Try realising instead that not every hammer fits to any nail.
Fixture Gallery 2.0
Gojko Adzic has released another major release of his Fixture Gallery. Personally I found this gallery valueable when making the first steps using FitNesse for our test suite rework. If you’re looking for something, take this as an additional source of information on how to create tests and the according fixtures. Gojko additional covers several programming language of Fit and the FitLibrary, which makes it additionally valueable for a large audience. Simply see for yourself.