Gunnar Kudrjavets

Past-due software projects, can something be done about them? (Part I)

Anyone who has ever read CHAOS report by Standish Group or participated in at least one software project knows that Murphy’s law works – if anything can go wrong, it will ;-) Like Fermat’s Last Theorem puzzled mathematicians for centuries, the problem of estimating correctly the time to complete any given software project has been extremely difficult to solve, although most of the people in software industry battle with it every day. In spite of the fact that Fermat’s Last Theorem has been finally solved and mathematicians are cheering, when it comes to delivering software on time, it’s still very hard to see the light at the end of the tunnel. Being also a chronic pessimist (though I rather prefer to be called a skeptic ;-)) I doubt also that we’ll find some silver bullets anytime soon. To summarize: when software engineering as a separate discipline was established, things were bad and they aren't much better now ;-(

But enough of being negativistic and let’s better take look at what mortals like you and I can do to keep the wheels of software industry running more smoothly. For myself I’ve set two simple goals:

1. Being able at any given moment to have an adequate understanding where the software project stands and where it's going. Is it on track? Is it late? How much is it late? What can be done about this? Who is doing something about this? Does it actually make any sense to strive for our initial dates? Do I see any potential disasters in nearby future? This has been proven to be extremely useful mainly to avoid cases like this: you have 6 months for accomplishing something; every single week everyone reports that they’re on track; a couple of days before the final deadline one of the team leads says that there’s a slight problem and it’s going to take at least three weeks to meet the final set of requirements. This is a variation of "I’m done with coding; now I only need to debug, test, fix some bugs, and refactor the code a little bit." Unfortunately situations like this happen more than anyone wants to admit ;-( Fortunately in my team there’s something that we call "management by no surprises" and believe it or not - we don’t shoot the messenger for giving realistic status updates. Otherwise I would have been demolished a long time ago (most of the news I deliver start with the phrase "There seems to be a problem with ...").
2. Being able to analyze, document, and understand all the issues which caused something to be late. What are exactly the things which caused us to be late? How did they happen? What was the root cause? How can we prevent these things from happening in future? For me this has proven very helpful to understand where the actual problems lay and do something about them. Please don’t confuse this with the interrogation style root cause analysis á la "Now we need to find somebody to point finger at and claim it’s all his/her fault! You, Alice, what do you think, whose fault it is?" I still believe that making any of the mistakes more than one or two times is the sign of incompetence and one should spend significant amount of time analyzing his/her failures and learn from them.

Attaining even only these two goals isn’t as trivial as it may sound. Because of being classical Virgo by birth, I have these strange habits of trying to categorize, classify, and label everything ;-) Here is the list of major categories of problems I’ve encountered while being responsible for completion of different software projects. I call them actually "different type of scheduling games software engineers play", mainly because of this book:

1. Game 1. Everything’s on schedule, but suddenly we need to do some additional work (add a new feature, own an additional area of responsibility, comply with some additional coding standards etc).
2. Game 2. Everything’s good in our team, but because of some dependencies or unforeseen events the project will take longer than expected.
3. Game 3. We tried to do our best while estimating, but made a number of mistakes and now it’s going to take longer than we initially thought.

I’ll write about my strategies when playing these "games" in the second part of this post, but first it’s essential to see what major possibilities there are dealing with software projects which are late:

1. Just move the end date. Nobody likes this solution even if it could be justified (market conditions change, the end date isn’t as important as it used to). Moving the end date is generally viewed as a sign of incompetence and admitting the fact that errare humanum est ;-)
2. Add more resources (people). Every decent engineer knows the Brooks Law - adding manpower to a late software project makes it later, but there are exceptions to every rule and one should use common sense to determine when it’s appropriate or not. I’ve seen the Brooks Law uphold in most of the cases, but I’ve also seen the cases where adding manpower has been extremely successful. For example: in case of our product we achieved reasonable amount of happiness with involving a developer from not our core team to develop some piece of GUI for us. If a person knows already ATL, C++, and Win32 GUI programming then writing isolated GUI isn’t so complicated task and it doesn’t require deep knowledge of how server code actually behaves. Of course this type of “outsourcing“ won’t work in case when we would take a person not familiar with our architecture and ask him/her to optimize the way our error handling model is designed ;-)
3. Make people work harder. You all know what this means. One morning you wake up and understand that there are no more clean clothes at home, refrigerator is empty, and you don’t know what’s been happening in world for last couple of weeks. I have possibly too much "Marine Corps" mentality in me, but I think that for the necessity of developing one’s character and world view every software engineer should participate in at least one Death March project. Believe me; it’ll make you love the time you’ll spend on estimating your project completion date next time ;-)
4. Cut features. Sometimes the end date is extremely important. You need to ship with some other product at the same time. You need to ship as a part of a bigger product. Shipping the product is extremely important to success of a company overall. Shipping a product is important to get first into the market.
5. Cancel the project. Refer to the CHAOS report mentioned earlier. Seems to happen quite often in software industry.
6. Lower the quality. I usually tend to say that either we spend an hour to fix some problem now or we’ll spend a week fixing it later. Everyone in software industry experiences this probably every day. Yeah, let’s not handle these error codes or exceptions because we know this error can’t happen, it takes some time to write this code, and if it happens then something is really wrong and our product wouldn’t work anyway. Two weeks later one of your key customer tries to install your product, setup terminates and doesn’t say anything. Key people are pulled off their projects, spend day to investigate the problem only to understand that bad things can happen. Fix is made and the new version is deployed to customer. Time spent: order of magnitude more than it would have been for Alice or Bob to write an error check which would help customer quickly identify the problem.

To be continued...

P. S. Though I consider watching TV as wasting time, I have to admit that premiere of “The Shield: Season 3” today was fabulous ;-)