Last week I sat in a meeting with a ProductOwner, a ScrumMaster, and the Development Team. The team works on a legacy code base with about 2 million lines of code together with 13 other teams. Thus far there has been little to decouple the various components. In that meeting the topic of refactoring came up. The bottom line was that the Development Team needed to ask the ProductOwner for time to refactor their code for each story.
What a waste of your time.
Personally, I believe that if you have to ask your ProductOwner to plan in time for refactoring, the ProductOwner should stop all work on the product, and send you on a class on working effectively with legacy code if you are an internal employee. If you are an external employee, your client should fire you immediately.
Wait, what? That’s harsh? I don’t think so.
Let’s take a look at the underlying system dynamics.
Prioritization
The primary responsibility of the ProductOwner is to prioritize the Product Backlog in order to develop the most successful product that’s possible with the given circumstances. Clearly, the ProductOwner optimizes the flow of backlog items to generate the biggest possible value out of the product.
If the team asks the ProductOwner to prioritize between technical polishing and business value, what would happen? First of all, in most cases I have seen, business value had the greater priority for the ProductOwner. Most of the times the ProductOwner was not even able to distinguish the value of refactoring items. (Most of the times, ProductOwners are not even able to distinguish the value of business items either, but that’s a different story.)
So, how would you choose if you could have 10,000 Euros now with a dirty kitchen, or a clean kitchen, and maybe 10,000 Euros later, what would you pick? Of course, the dirty kitchen could be a bit short-sighted. That’s probably why good ProductOwners understand the necessity for polishing work. The problem is that good ProductOwners do not fall from heaven. They need to learn. If you have a learning ProductOwner, you shouldn’t have him prioritize between polishing work and business value.
Technical Debt
The primary responsibility of the team members is to deliver a technical sound product. At the end of the iteration, the product should be potentially shippable. Scrum teams have to face the perfection challenge: deliver something of value fast.
If you leave out necessary refactoring, you introduce a delayed feedback loop into your system that is clearly not necessary. If there is refactoring work, you should not put that work in a backlog – either obviously or non-obviously by delivering a lower estimation under this or that condition for leaving out the refactoring step. That would be accumulating technical debt. Your ProductOwner needs to pay for this debt later. Oh, you will also have to pay for that debt later as well.
The problem with technical debt is the delay introduced in the feedback loop. The more and more technical you accumulate, the more the necessity for the grand redesign in the sky grows. On the same note you are slowly boiling yourself by introducing more and more debt that slows you down.
Over time, you will deliver new features slower. This will either lead to the ProductOwner trimming the tail sooner since the product generates less revenue than necessary. Or you can convince your ProductOwner of a refactoring sprint. I have seen this a couple of times. What refactoring sprints mean in fact is that you haven’t build the best possible product before. Of course, there are blind spots for all of us. In my experience refactoring sprint provide an excuses for the Development Team to not deliver the best possible product. In the end you will find yourself with more technical debt than is really necessary.
Instead, consider challenging your team to learn to really deliver an increment of working software. The ProductOwner should be able to say “ship it!” during the Sprint Review, and it’s live two seconds later. Ok, maybe ten seconds, but no more. No “wait a minute, there is this thing” or “oh, we really can’t do that now”. Thereby you help your ProductOwner to fill his role without delays by providing the necessary transparency of where the product really is.
Empirical processes
The whole story boils down to empirical process control. Empirical process control consists of three major elements: inspection, adaption, and transparency. Inspect & Adapt underlies Scrum in large portions. Without it you wouldn’t be able to improve over time.
But Inspect & Adapt without transparency leads to erratic improvements. If you try to tune a system that you don’t have the relevant information from because people are hiding critical information like how good the product really is, leads to improvements that could eventually increase the demand for refactoring.
Your ProductOwner will only receive a complete picture for the underlying problems in the software, if you can provide her full transparency about the problems that you know of when introducing. You are not delivering the best job you can, if you avoid that.
Also note that technical problems that lead to fewer business functionality is not something your ProductOwner should worry about. That would be similar to a taxi-driver asking a passenger for the best route to take in a foreign city. That’s unlikely to work out – if the passenger does not know the city.
The same holds true for most ProductOwners when you ask them to decide for priorities of technical solutions when they don’t know the code base well enough to estimate the risks around that. In most implementations that I have seen that ProductOwner couldn’t know because he had other daily business to turn to.
So, clearly, as a member of the Development Team it’s your responsibility to be able to deliver the best product. Your ProductOwner is unlikely to know about the technical trade-offs that you put in your estimates, and the amount of technical debt you accumulate when rushing through the code base, or the amount of legacy code you have to tackle with.
Instead, provide a reasonable estimate for the work you see, and have a chat with your ProductOwner on the why. In the end, consult with her in order to split too large items smaller – but avoid cutting off the “refactoring” in a separate part. Your product will thank you in the long run, and you will deliver a more professional job by doing so.
I do not think asking time for re-factoring to a product owner is a bad idea . I agree the PO is a business person and might not understand what Tech Debt is , but as a software delivery team you need to give some idea to the PO about the trade off in accruing Tech Debt .
I have seen (At-least few times ) , where the dev team has convinced the business to takes time out of delivering new functionality to work on Tech Debt , and we as a team are better off in the longer run because of that (i.e Not finding a lot of bugs after each small change).
I assume you understand what Tech Debt is . Being a Tester following is a article I found useful to understand Tech Debt
http://www.codinghorror.com/blog/2009/02/paying-down-your-technical-debt.html
-Debashish
Speaking of firing someone… What´s a reason for a PO to get fired? Well, I´d say, if he asks for an estimate at all. Because any estimate – even if not intended – works like a metric. That means devs try to deliver on the metric – and hardly anything else.
As for the devs and refactoring: I´d call it not very professional to ask for time for a refactoring. That´s just what has to be done, and so it should be done without permission. Like writing tests does not need permission or designing a solution before coding.
If there´s a ton of legacy code, what happens is, features get delivered slower and slower. But always at the same level of inner quality (ie. evolvability, correctness). That´s what professionals do. They do not compromise sustainability for short sighted wishes. They just tell a customer: This is what we do. This is what you´ve hired us for in the first place.
And if the customer (PO) tries to overrule such professionalism… well, then that´s a sign of distrust and/or ignorance. (Of course except for rare cases where inner quality really needs to be sacrificed in favor of some other value for a short period of time.)
That said: If the situation is like this – thought about firing someone just because he wants to do the right thing, people not trusting each other, professionals being silent about what they are doing etc. – then there´s something bigger lurking in the dark.
And by the way: I´ve very, very rarely seen any true PO. There are lot´s of teams calling themselves agile, doing Scrum one way or the other… but a true, dedicated PO, close to the team, pulling at featrues… that´s a rare beast.
So, please, don´t put too much blame on devs.
-Ralf
As a PO I find it ok if he asks you for an estimate because he does not know about a better way for release planning or coordinating with other teams, like marketing. The question to challenge that believe usually is whether you are in the position to provide an alternative.
Thanks for triggering another idea around negotiating the psychological contract as a programmer.
Best
Markus
I see more and more projects providing full transparency to their PO about their Technical Debt. They use SonarQube and the SQALE indicators for discussing and analyzing the situation with the PO. Because indicators report about amount of TD, level of the risk, type of impact (potential bug, less changeability or worst maintainability….). ) it is understandable by almost every PO. Also because it works in the delta mode ( new and removed debt between versions it support pragmatic decisions about paying off (or at least some part) TD.