Phil Kirkham suggested in the comments on my entry about Testing and Management mistakes to do a write-up on how Tim could have responded in a better fashion. Personally, I decided to show to fashion of this: the incongruent style and the congruent style. Today I will begin with the incongruent one, providing a more congruent view in a later blog entry. Just in case you haven’t read the source of inspiration from Michael Bolton, yet, you can read it here.

This is a response to a recent blog entry from Michael Bolton. He challenges to spot as many management mistakes as possible in a short dialog excerpt between a project manager and a tester. I will quote the dialog and walk through each of the problems I can spot. Though asked differently, I will not only mention those problems related to management, but also to the tester there.

Magnus the Project Manager: “Hey, Tim. Listen… I’m sorry to give you only two days notice, but we’ll be needing you to come in on Saturday again this week.”

Tim the Tester: “Really? Again?”

The manager does not give the reason for this action. He simply states there is the need for the tester. This sets up emotional pressure on the tester, who does not want to let the team fall. I’m not sure if the manager put this up by intention or just by coincidence.

Tim on the other does not start to challenge the claim. The project manager simply says they need him, especially. But maybe another tester could do the job. What’s his mission? How long should he test? To whom should he report his results? Who is the client? Instead, he simply asks from the personal perspective, which might indicate an emotional reaction to the claim from the project manager. So, let’s look how the manager responds to this.

Magnus: “Yes. The programmers upstairs sent me an email just now. They said that at the end of the day tomorrow, they’re going to give us another build to replace the one they gave us on Tuesday. They say they’ve fixed another six showstoppers, eight priority one bugs, and five severity twos since then, and they say that there’ll be another seven fixes by tomorrow. That’s pretty encouraging—27 fixes in three days. That’s nine per day, you know. They haven’t had that kind of fix rate for weeks now. All three of them must have been working pretty hard.”

The build is a replacement for a broken build. The programmers seem to have produced some lousy work the last time. Now, that’s interesting information. Despite this valueable information, the manager seems to rely on the facts as counted by the numbers. There seems to be some corporate metrics program, that let’s this manager look more particularly on the number of bugs fixed than how many further bugs could have been introduced by fixing those. Again, this information is available in any bug database, but the manager simply ignores this valueable information. Last, but not least, his interpretation in the last sentence indicates that “more bugs fixed” means “harder worked”. This mustn’t be the case. Another point that Jerry Weinberg made me aware of, is the fact, that there is nothing stated about how many incidents correlate to the mentioned bugs. There could have been 10 bugs fixed by a single code line change. Similarly there could be a single bug which needs fixing in several components. The numbers alone lie here. A final point: The manager should know if his programmers have worked hard , if he would have paid attention to what they were doing rather than looking at the numbers, only.

Tim: “They must have. Have they done any testing on those fixes themselves?”

Tim clearly should know better here than to give in to the manager. The developers mustn’t have worked very hard as pointed out before. Tim could know this better, if he paid attention to the past of the project. Just by stating that the only outcome here is that the developers were working hard, he proves his manager correct – in the manager’s eyes. If you believe something hard enough, your mind makes it a fact. This is inattentional blindness.

Magnus: “Of course not. Well, at least, I don’t know. The build process is really unstable. It’s crashing all the time. Between that and all the bugs they’ve had to fix, I don’t imagine they have time for testing. Besides, that’s what we pay you for. You’re quality assurance, aren’t you? It’s your responsibility to make sure that they deliver a quality product.”

Of course the developers did not test, how could they deliver that many fixes otherwise? There are several pointers that this managers ignores relevant information. He doesn’t know if they tested their code, the build prociess is unstable, but that doesn’t give him a clue, either. In combination with the Assurance Fallacy of Testing (“You’re quality assurance, … It’s your responsibility…”) there really is more need to fix bugs, than to fix them properly. Of course, the build process crashing all the time could indicate a problem in the quality of the code these programmers seem to produce.

Tim: “Well, I can test the product, but I don’t know how to assure the quality of their code.”

When someone is arguing irrationally, don’t try to argue with logic. “I know that you have many duties as a project manager, but doesn’t the broken build process point out, that there might something wrong?” could be a response to get the attention of the manager back to quality rather than delivery. The hidden message Tim wants to send here just doesn’t work in this context.

Magnus: “Of course you do. You’re the expert on this stuff, aren’t you?”

Without knowing this particular project manager for sure, I sense that he is basically just trying to sell the idea on testing to this poor tester. For any reasonable tester it’s obvious that he does not know anything about testing at this point. Instead, by giving kudos (“expert”), he tries to let the tester feel comfortable. This is the “washing machine selling management”. In this style management acts as if they were selling you a washing machine. Here, the selling involves the tester to be the right person to do overtime on the weekend. Anyone who thought about testing at least a tiny bit, should get to the conclusion, that testers don’t assure any code quality. But as pointed out, the manager simply wants to get this tester to work over the weekend by sacrificing even his own credibility.

Tim: “Maybe we could arrange to have some of the testing group go upstairs to work more closely with the programmers. You know, set up test environments, generate data, set up some automated scripts—smoke tests to check that the installation…”

This is a brilliant idea. Let testers and programmers work together to fix the team problem, help programmers get started with unit testing and the like. Too bad his claim is interrupted here by the manager, thereby undermining the testers management capabilities. Of course, the tester shouldn’t make suggestions on how to manage the project here.

Magnus: “We can’t do that. You have high-level testing to do, and they have to get their fixes done. I don’t want you to bother them; it’s better to leave them alone. You can test the new build down here on Saturday.”

At this point, the manager fires back by telling the tester how to test. Personally, I think this is a response to the tester’s attempt to manage the project. Of course, the fallacy of testers interrupting and interfering with the development is inherent here. Testers help developers get going. If the priority is to fix the bugs, rather than to fix them properly, then it’s more important to get some software. If the software does not need to work, I can hit any other requirement (The Zeroth Law of Software taken from Weinberg’s Quality Software Management Volume 2).

Tim: (pauses) “I’m not sure I’m available on Sa…”

Magnus: “Why not? Listen, with only two weeks to go, the entire project depends on you getting the testing finished. You know as well as I do that every code drop we’ve got from them so far has had lots of problems. I mean, you’re the one who found them, aren’t you? So we’re going to need a full regression suite done on every build from now until the 13th. That’s only two weeks. There’s no time to waste. And we don’t want a high defect escape ratio like we had on the last project, so I want you to make sure that you run all the test cases and make sure that each one is passing before we ship.”

Noticed that this manager does not even let the tester finish? Again? This indicates that the manager does not have any interest at all to get to know what the opinion from his tester is. It’s more important to have him working on Saturday. Opinions do not count. In addition. he puts more pressure on the tester here. The schedule is of course pressuring this manager, but it’s his job to manage to get a quality release.

In addition his claims in this response are not congruent to his earlier message. He cares about quality from the tester, but not about the quality from this who arrange quality in the code itself. This is ridiculous and I really do not want to be in Tim’s situation at this point. The manager again tells the highly-educated tester what he has to do. Mary Poppendieck explained to me that this is really a problem.

Once again the manager is ignoring the facts. Every “code drop” has had problems in the past. So, where are the code reviews? Where are the unit tests and walkthroughs? Where are the design reviews? Given the information the manager seems to have, he could really do better.

Tim: “Actually, that’s something I’ve been meaning to bring up. I’ve been a little concerned that the test cases aren’t covering some important things that might represent risk to the project.”

This response again is good, but he could do better. Telling the manager that the problem indeed might be in the way he manages the project with the information at hand, is ridiculous. The Candidate Product Rule says, “Actually, it ain’t nothin’ ’til it’s reviewed” (Weinberg, QSM Vol. 2, page 290). This manager seems to manage by the rule, that it’s nothing until it’s tested, but we can skip review easily. The claim Tim makes here is still valid, but he doesn’t respond to the things Magnus just raised. He could bring his point out more clearly by doing so.

Magnus: “That might be true, but like I said, we don’t have time. We’re already way over the time we estimated for the test phase. If we stop now to write a bunch of new test scripts, we’ll be even more behind schedule. We’re just going to have to go with the ones we’ve got.”

Ah, the common time creation myth. Time is not created, you have to take your time. Especially for quality products there is no free lunch. You have to plan accordingly, incorporating the time it takes. The problem that the project is already over estimate, might as well depend on the manager of the project. And the manager is falling for the test phase fallacy. Testing is not a phase, bugfixing might be. Last, not adapting to the project as it unfolds by creating new test cases may as well produce the problems of tomorrow. This vicious cycle the Magnus manages to get in here is very common.

Tim: “I was thinking that maybe we should set aside a few sessions where we didn’t follow the scripts to the letter, so we can look for unexpected problems.”

Magnus: “Are you kidding? Without scripts, how are we going to maintain requirements traceability? Plus, we decided at the beginning of the project that the test cases we’ve got would be our acceptance test suite, and if we add new ones now, the programmers will just get upset. I’ve told them to do that Agile stuff, and that means they should be self-organizing. It would be unfair to them if we sprang new test cases on them, and if we find new problems, they won’t have time to fix them. (pause) You’re on about that exploratory stuff again, aren’t you? Well, that’s a luxury that we can’t afford right now.”

Well, the claim from Tim to do some exploratory testing to fill the gaps for the uncovered areas is great. Magnus’ response to it is fatal to the project. Again the project manager does not adapt to the situation at hand, after the project started to unfold itself. By getting risk-averse coverage of those areas which are not covered by the initial test cases, the manager sets himself up for tomorrow’s problems. Oh, and of course, it’s better if the customer finds the bugs, rather than have the programmer fix them now before the loss of reputation. Last, but not least, exploration isn’t a luxury, neither is testing itself. They’re both a necessity for quality software. Skipping either or both is not only undermining testers’ efforts, but also comprising developers who very well observe what managers do and don’t do. Finally, the manager has misunderstood Agile in the sense to develop software without the weight of documentation, but has not understood that Agile development needs a team. By separating the testers from the developers the manager undermines every benefit he could get from really doing Agile development.

Tim: (pauses) “I’m not sure I’m available on Sa…”

Magnus: “You keep saying that. You’ve said that every week for the last eight weeks, and yet you’ve still managed to come in. It’s not like this should be a surprise. The CFO said we had to ship by the end of the quarter, Sales wanted all these features for the fall, Andy wanted that API put in for that thing he’s working on, and Support wanted everything fixed from the last version—now that one was a disaster; bad luck, mostly. Anyway. You’ve known right from the beginning that the schedule was really tight; that’s what we’ve been saying since day one. Everybody agreed that failure wasn’t an option, so we’d need maximum commitment from everyone all the way. Listen, Tim, you’re basically a good guy, but quite frankly, I’m a little dismayed by your negative attitude. That’s exactly the sort of stuff that brings everybody down. This is supposed to be a can-do organization.”

If it’s natural for Tim to come back all the weekends, he will be asked again. Of course he shows his struggle during this conversation, but it’s really for the manager to influence to get what he particularly wants him to do. Again, in the example stated, the manager ignores very well information which is available to him beside the information he gets from the testers. The pointer to “bad luck” just reminds me on Weinberg, who stated to replace the word “luck” in the phrase with “management”, when a managers refers to bad luck. We might as well prey to the software gods, if a whole company can be built upon management by luck. Of course, the manager’s claims about commitment and can-do organization are just institutionalized to make his argument and get Tim motivated. The problem lies in the decrease of motivation after this conversation. Tim’s trust for proper management has surely decreased, and a few more of these instances may let Tim leave the company searching for a better job opportunity.

Tim: “Okay. I’ll come in.”

Finally, Tim gives up. Never do this, though it might mean to continue fighting for ages or to lose the job. This is ok, as you didn’t like the job, anyways, now do you?

That’s my more elaborate response to Bolton’s challenge. Now, which parts did I miss?

Today, I started off a series of documentation fallacies. Gerald M. Weinberg inspired this series. Since I didn’t have the initial inspiration quote handy at the time, I looked it up right now. As it is the initial one, I call it the documentation fallacy #0. It’s a citation from the reminders in Perfect Software… and other illusions about testing.

The Documentation Fallacy #0:

Believing that the mere existence of documents has some value.

Just because there is a document, means nothing. The document might just simply plainly lie around and start to get dusty. When no one is reading your document, then you have basically wasted all the time that you spent on preparing to write it, write it, review it, publish it. Period. Well, maybe you learned something as a side-effect, but you could have learned that one without writing a document that no one reads in first place. Ok, maybe you could have learned something by writing your thoughts down on the learning topic, sure, but still, the writing isn’t worth the paper that it is going to full. Save the rain-forest, remember?

So, just writing a document means nothing. It could be of no value because it has low quality, it could be of no value as it describes the life of Alfred E. Neumann, or it could be of no value as it describes how to the 8086 CPU was invented. If your project is not about any of these, you will simply not care about the content. More dramatically, your document could distract the people on your project and mislead. In the examples before I exaggerated to make a point, but what if the document was plainly wrong, and people started to work after it? Maybe you will end up with a project way behind schedule when you find out the problem right after delivery of the system. Doesn’t sound good, does it?

The Documentation Fallacy #1:
Thinking that more documentation means more thorough.

More documentation doesn’t mean anything. Personally, I find great value in those documents, that are as crisp as possible and point me out where to find the details if I need them. Usually I started to dig into a topic by getting a rough overview. After that I start to dig deeper and search for more details. This is the reason why to write requirements in a rough form before the work on the implementation begins. This concept can be found in use cases. Goals build the high level overview. There are different goal levels, and for more complicated business flows you write down an overview use case description alongside with finer-grained use cases on the subsystem or function level as you may see fit. Similarly when working on a user story, you start with a high-level overview and then start to dig deeper.

So, starting with a crisp overview is great. But when digging deeper you don’t need to document everything. The trivial cases are mostly irrelevant to document, because everyone already knows what shall be done in the login use case for example. Well, maybe you have a more complicated login use case, then document it, but if it, refuse to document the very obvious. Simplicity – the art of maximizing work not done – is the key here. But this does not mean that you shouldn’t document anything at all, of course. That would be stupid.

The Documentation Fallacy #2:
Thinking that since it’s documented, it gets read.

Just because it is written down somewhere, does not mean that anyone will read it. Maybe you’re lucky enough that someone reads it, but you can’t rely on it. Similarly you cannot rely on the fact that the people who actually need to read the document, have read it. Most of the time, documents simply lie around with no on reading them, covering dust – even virtually on the hard-drive of your computer. In the waterfall model we introduce the biggest problem here, since the documents don’t necessarily get read. I remember a case where we run into a problem during the user acceptance testing resulting from customer stakeholders not having read (properly?) the documents that we had sent them. Obviously this was the wrong time to find out that the software didn’t realize the functionality properly.

The Documentation Fallacy #3:
Thinking that since it gets read, it gets understood properly.

The Satir Interaction Model taught me, that not necessarily everything that gets read is understood in the same way for all readers of the document. The model splits up communication into four phases: intake, meaning, relevance, and response. During the intake each person takes in some parts of the message that she just heard. Therefore it gets distorted during the reception already. The meaning phase then interprets the message based on the personal rule model and previous knowledge of the receiver. This might distort the message even further. Overall, this may result in necessary details left out, or unnecessary details added to that message.

That said, it’s obvious that everyone may understand something different when reading a particular document. As the message and the meaning are distorted from what the receiver takes in (partially) and what she adds to the message based on previous experience, the understanding may well end up as being something completely different. So, just barely reading a document does not mean that the author’s intentions were understood.

The Documentation Fallacy #4:
Thinking that since it gets read, it actually gets used downstream.

Thanks for Paul Boos, who added this to my initial list. The two parts from the Satir Interaction Model left out of the discussion so far are the relevance and the response parts. On the relevance I assign to the distorted message how important I think that interpreted message is to me. If it is irrelevant, I might end up ignoring the message overall, thinking “fine, so what?”, or “don’t bother me.” I might also end up giving the relevance to completely ignore the message without responding to it. In the response I can decide how to react to me interpreted message, that I assigned some relevance. I can say something completely different, even out of context, like “nice weather outside, isn’t it?” This means that the mere existence of a document does not mean that I will follow it’s contents when reading it. It’s important what relevance I assign to the message and how I decide to respond to that message.

The Documentation Fallacy #5:
Thinking that since it is understood properly, it is used properly

Thanks for Jeroen Rosink for this addition. As derived from the Satir Interaction Model again, the response I take to my understanding of the document may vary to a great deal. This means that I might understand the document, but this does not necessarily mean that I will apply the message accordingly. In fact, most of the software I crossed was used unintentionally, in different ways, that the developers initially did not intent. The Windows scripting capabilities are one example of this coincidence, but there are many, many other examples out there. I’m sure you can name at least five on your own.

Please let me know of any additions I could make to that list. Hopefully, I raised some of your thoughts by now, so share your stories on misused or unread documentation with me.

Recently I finally finished the fourth volume of Jerry Weinberg‘s Quality Software Management series. As a wrap-up I decided to write a summary of what I learned and why you should read it yourself, too, if you haven’t already.

Introduction

In summary, the first book introduces into the cultural patterns that Jerry has found during his work with software companies. These cultural patterns include the Oblivious (Pattern 0), the Variable (Pattern 1), the Routine (Pattern 2), the Steering (Pattern 3), the Anticipating (Pattern 4) and finally the Congruent (Pattern 5) culture. He then digs deeper in the first three volumes on how to create a Steering culture. In the fourth volume he goes into the details to reach an anticipating culture.

Jerry provides a compelling view on the act of creating software, testing it and delivering it. Interestingly his view is right up-to-date, although he wrote the four books more than a decade ago. Next, I’m going to get into detail on each of the key lessons I took with me while reading through them.

Quality Software Management Volume 1 – Systems Thinking

In the first volume Jerry provides the cultural pattern model and relates it to the CMM levels and other approaches taken. Interestingly his view includes the culture around the people actually producing the software in first place, while other models focus on anything else but the people. First and foremost, Jerry defines quality as “value to some person”. Based on this definition, the question we have to ask in software development – which I define as programming, testing and delivering – is whose values counts most to us.

Additionally Jerry introduces the reader to systems thinking. That is a model to think through difficult and maybe complex situations in software projects. He shows many graphs applying systems thinking, thereby creating deep insights into each of the cultural models, as well as the dynamics around the creation and maintenance of software.

Using systems thinking Jerry introduces the non-linearity of aspects in software development. He revisits Brook’s Law and generalizes it to hold not only for late project, but for any project, when adding people, you’re making it more complicated, and maybe even later.

Jerry provides a compelling view on software error, distinguishes between faults and failures, and thereby provides a model for measuring, which is the topic of the second volume.

Quality Software Management Volume 2 – First-order measurement

In the second volume of the series, Jerry continues to dig deeper into the cultural model of software development organizations. Jerry presents a thorough discussion of the Satir Interaction Model. The Satir Interaction Model splits up communication into four major steps: intake, that is how I take in information; meaning, which is derived by making interpretations from the received message; significance, that is whether I start to bother or not; and finally the response on the initial message. All these steps happen in a fraction of a second in human communication.

The Satir Interaction Model is useful in situation where you start to make meaning too soon based on fractions of the data that you got. Often, this leads to mis-interpretations about the intentions of your communication counter-part and therefore not only to a poor dialogue, but also to a big misunderstanding. That’s why Jerry raises the Rule of Three:

If I can’t think of at least three different interpretations of what I received, I haven’t though enough about what it might mean.

In parallel to the discussion of the Satir Interaction Model, Weinberg continues to show the relevance for first-order measurements in management. Using systems thinking as introduced in the first volume of this series, he explains management actions and their results. He continues to dig into metrics that are useful for managing. Finally, he discusses zeroth-order measurements, that each management should be aware of. These measurements tell you on which things to keep an eye in order to manage your software project.

How to act upon these data is the topic of the third volume in this series.

Quality Software Management Volume 3 – Congruent action

In Congruent action Jerry introduces congruent communication. For any effective communication one has to respect the self position, the other position and the context about the communication. When leaving one or all out, you’re going to find yourself in an incongruent communication style. Leaving out the self-position leads to placating behavior, missing the other-position leads to blaming, leaving self and other out is a super-reasonable response, finally leaving all of them out is just an irrelevant coping style.

Additionally Jerry discusses the Myers-Briggs type indicator system. He introduces the four dimensions in that model. The first of the four letters says something on how I refresh my energy – introverts (I) for seeking self-reflection or extrovert (E) for interacting with people. The second letter describes how I take in information – either by facts from the sensation type (S) or by grasping the abstract concepts via intuition (N). The third letter says something how I make meaning – by pure logic in the thinking preference (T) or by grasping the feelings of the humans around me (F). The fourth and last letter states how I prefer to take actions – the judging (J) style prefers to settle decisions, while the perceiving (P) style prefers to leave options open. So, as I’m an INTJ, I prefer to take energy from being alone, intuitively grasp the abstract concepts, making meaning of them through logical thinking, and prefer to have things settled.

So, a congruent manager should consider their preferences and see the preferences from others using the Myers-Briggs type indicator model. Whenever I am aware that my message was not receipt correctly at my communication partner, I can take the choice to reflect and maybe present the information in a different format for my audience.

In addition Weinberg concludes in this third volume in the series how to reach a Steering culture built upon systems thinking, first-order measurements and congruent responses while leaving the transition to an anticipating culture for the fourth and last volume.

Quality Software Management Volume 4 – Anticipating change

In the fourth and last volume Jerry introduces the Satir Change Model. Since an anticipating culture relies heavily on change artists, this model comes in handy in forming such an organizational culture. Weinberg explains in great detail how people react upon foreign elements in the organization, and that too many threating external influences may result in people hiding in their basement. In addition he walks the reader along the change model and explains how to get from an old status quo to a new status quo and actually make and foster the necessary change.

Weinberg continues further how an anticipating organizations works through the overall software development process. Starting from meta-planning, tactical change planning, over to planning as an software engineer, this volume conducts how to make change stick in the organization. In addition he writes about process and process improvements. He lists processes commonly in practice by the time he wrote the book (1997), and discusses why it is important to know several process models as a change artist. Weinberg describes the differences between a process vision, the process model, and the process.

Finally, he gives a compelling view on things that are still relevant nowadays. Taking a closer look on how to terminate projects, and how to know that you should terminate or re-plan them, he gives a thorough overview of do’s and don’t’s in software development for the project manager. He discusses that requirements documents and design documents should be placed in a version control system alongside with the code. Considering that he wrote this book more than ten years ago, it struck me when I encounter multiple teams which are still not using any version control mechanism at all. Finally, Weinberg takes a look on tools, and how and when to introduce them.

The biggest gift in this volume are the lists of do’s and don’t’s in software projects alongside with the eleven commandments of Technology Transfer. His comparison of the waterfall model to other software development models may be seen as a bit outdated, but most of it is still relevant. Weinberg even touches the spiral model in this discussion, so portions of it also apply for modern Agile methodologies. Of course, the principle to know about many models and to know when they apply and should be used, is the key lesson to take away here.

Summary

Overall, Jerry provides many techniques and models, which come in handy at understanding the dynamics at play during a software project. Systems thinking helps you to analyze complex social systems in your project. The cultural model provides you with a useful view on how quality is seen in a particular organization, and which steps you might take for improvements. The Satir Interaction Model is useful in critical project situations, where your own measurement system is likely to be collapsed under the pressure of the situation.

Last, but not least, I’m glad that I got my copy from QSM Vol. 1 signed by Elisabeth Hendrickson. It was a coincident during last October, but she told me she felt honored to sign it for me.