Today I came across a blog entry on the need for programming expertise for test automation. I had the urge to reply. While doing so, I decided to put my reply up on my blog and call it the tool vendor fallacy. Just in case…
I don’t quite agree to your statements. The problem is not whether your tool should support you to do some script, or you need help from a programmer. The problem stays the same, but has different stakeholders.
You might claim that a tool will support your testing needs in better ways as compared to a home-grown solution. The problem is, that you put the responsibility to test the code that does your test automation to the automation vendor.
Instead of building your own solution with everything you need, you ask the tool vendor to make assumptions for whatever you will need in your product in ten years to come. He might be good at guessing your need, or he might fail to predict the future. He might actually build a great tool for you, but he might also fail to include this tiny little bit that your whole business will rely on.
The problem is not whether or not you need to program to automate tests. There will be always some code involved in building automated tests – may it be the tool or your own home-grown code. The problem rather is that you lack support of programming experts to help you with automating your tests.
This is a problem you will have to solve on your own. There is no tool vendor necessary to solve that problem.
Markus, thanks for reading my blog and commenting ( http://bit.ly/ji8h7e ). I am actually glad to see that we are not that far apart in our opinions. I absolutely agree with the need to have the ability to code in order to build a full-scope, commercial automated tests.
It all comes down to the ages-old question of “build vs. buy”. If the only correct answer was “build”, people would still either be coding in machine codes, or in Assembly language at best, or had to develop their own language specific to their organization’s needs. To my opinion, all it helps to is the job security for those who do that. There were reasons why we now have integrated IDEs, 4GL tools, and other S/W development productivity tools. Nobody can argue that that’s the necessary eval to have to improve the speed of development. Using custom-build frameworks means that you can’t use the open-market resources, the support, the talent, and the expertise, to easily implement 95% of the tasks (development or QA).
For 5% of the absolutely-unique tasks you will always need custom programming. And a good tool should provide for that – integrate with Selenium, JavaScript, or other commonly-used technologies.
For the remaining 95%, a good tool should do the job. Why, please tell me, do you need to know the syntax of if-else, do-while, etc., programming constructs, and use a wrapping language to implement them for your tests? Why can’t the platform help you do that? Why do you need to write your database access statements by hand? Why can’t you have built-in browser functions, time processing, XML parsing, file access, etc., and still be proud of a well-written automated test? And why, in order to access a page object’s properties, you need to write a low-level function instead of selecting it from a drop-down list of the tool?
Can it be custom-build for your specific purposes? Absolutely it can. But why? Why do we trust vendors to build products for us? Because they have the expertise, the experience, the customer feedback, and the resources to make it right the first time. That’s why I prefer buying my BMW, not building it…
Hi Vadim,
in the way I use test automation tools, I start writing an acceptance test. Then I get to the task to automate that test. While automating it, and actually writing code, I discover the domain of the application that I am going to implement. By driving my automation code, I drive the application code, often using TDD in between as well. I got an example in my upcoming book where I do just that. A colleague of mine told me that he uses this approach all the time. I have not seen a tool which provides me this ability using built-in testing tools.
I think this is where we disagree.
I still use a hammer and screwdrivers, but I replaced my brace-and-bit with an electric drill many years ago, and I often use an electric saw instead of a handsaw. I am an amateur electrician, so I actually understanding these devices, but I choose to focus my time energy on woodworking rather than building my own power tools, and buy them instead.
I have *never* been a “car guy”. I have a theoretical understanding of most of the parts of cars, but in my experience, those parts don’t function anything like their theoretical counterparts. If I had to build my car from scratch in order to have transportation, I’d probably walk instead.
There’s no fallacy there, there’s just balancing abilities, priorities, and goals. As Vadim points out, there are a lot more bad programmers than good programmers. It’s been that way from the beginning and I don’t see it ever changing. Providing tools that allow bad (or non-) programmers to do useful things that are within the scope of their abilities is just the same as not requiring everyone to build their own car from iron ore, rubber tree sap, silica, and crude oil.
I have a problem with your car-driving analogy. The car-driving analogy holds just to a certain point. We don’t transport persons in cars, we program software systems. These software systems are usually more customized to the needs of our particular clients and customers. We don’t create customized human beings and drive them with cars, and this is where the problem steps in. In fact, most enterprisy systems I came across were indeed more like a whole tribe of over-customized robots that someone tried to transport using a bicycle.
The hammer, drill, and screwdriver analogy comes with its own limitations as well. Usually you have one tool for a special purpose in the real tool world. An electric saw does not limit you to cutting 10″ long wood. But that’s the limitation I saw most tools in the software development world come from.
Don’t get me wrong, I also like to use tools. But I also like to add my tiny bits of automation code that provides me freedom for my domain code as well. There is not an either or, but a vital mid. Looking at tools like FitNesse, Cucumber, and Robot Framework, these tools provide me with everything I need. It limits my thinking about requirements sometimes, but it helps me discover the domain that I am working with. Instead of providing me with tools that I might use – just in case – I can limit my work on the things that I really need. Like a screwdriver these tools help me find the screws to make my application robust – without getting in my way.