The wrong question

(This is part #3 in my mini series of blog posts about estimates. Previous posts: part#1 part#2)

It’s time to let you in on The Secret. Don’t tell anyone, but “How long does it take” is the wrong question.

I planned for this post to be about statistics and how to calculate probabilites. Instead it turned into a rant, of sorts. You see, there’s another Secret buried here: If you really take the time to learn how to estimate risk correctly, this will help you learn the following fact about the outcome of your project: It is very uncertain. I guess that’s helpful. Sort of.

“That’s just baloney,” I can hear some of you say. “My projects are consistently on time and on budget,” you continue. Maybe so. Let’s examine the famous “project triangle” for a moment.

405px-Project-triangle-en.svg

Ever wondered why it isn’t a square? Because quality is a result of the other three, some might say. “That’s just baloney,” you can hear me say. I think it’s because cost, scope and time is easy to measure and easy to adjust. Quality? We always deliver perfect quality! We are professionals! Let’s just put that in the middle, and hope no one pays too much attention.

You, the guy who screamed “We always deliver on time and on budget!” a moment ago. How about scope? “Hah! We delivered all the functionality as well! All the developers bitched about them having too much to do in too little time, but my project plan showed them wrong!”

That’s impressive. But even when we go to the painstaking length of figuring out the risk and uncertainty of a project PROPERLY – You know, with Statistics and stuff, not just, God forbid, guessing or anything – it still comes out as being very uncertain. So how come you can consistently deliver your projects on time, scope and budget? Your team don’t have much choice, do they? They have to reduce quality. Luckily for you, no one will notice until the project is over. And they couldn’t really measure it if they did. What IS quality anyway? Sounds like a made up word to me.

 

So what did I mean by “how long does it take” being the wrong question?

There are a couple of problems. First of all, what is this “it” that we are doing? When you start the project, odds are neither you or the customer knows what the end result is supposed to look like. You may think that you do, but you don’t.

If it is a waterfall project, the customer will say “This is what I want” while dropping the dreaded Complete Requirement Specification on your desk. Then you go away for three months with a team of developers to create whatever is in the specification. “Here is your product!” you exclaim with great enthusiasm when you come back. After the customer has tested it, she goes “Nope, that’s not it.”

If it’s an agile project, your team of developers only go away for maybe a couple of weeks at the time. Each time you come back and show a little bit of the product, the customer tests it and goes “Nope, that’s not it either. And where’s the rest?”

However you go about it, both you and the customer learn a lot DURING the project. Mostly what they don’t want. Needless to say, this process takes quite a bit longer than just doing stuff once. No one knows what done looks like until, well, you’re done. And please remember, we’re not BUILDING a product, we’re CREATING it. The first of its kind. Your developers aren’t packing meat in boxes along a conveyer belt, they’re freaking SCIENTISTS! So stop pretending you are a production facility. The product you’re creating has never been built before. If it has, please go and buy that instead.

And that’s not even half of it. What did you say were doing again? A project, right? Well, most of the time we’re not really creating projects, we’re creating PRODUCTS. The funny thing about products is that they don’t end when the project ends. That’s kind of when they start. In terms of cost, that means roughly 90% of the total cost of a product comes AFTER the initial release. Maintenance, bug fixing, additional development, paying back technical debt, people spending most of their day swearing over how horrible it is, and so on.

 

Here’s the Third Secret: The less time you spend on quality during the “project” phase, the higher the total cost of the product.

So whenever you pat yourself on the back for delivering “on time” (usually a random point in time decided by anything except the actual amount of work that needs to be done before that date), that probably means you have reduced the overall quality of the project, and increased the total cost of the product you delivered. That doesn’t really sound like back patting performance, if you ask me.

Did you think this series of posts would culiminate in How To Estimate Accurately In 3 Easy Steps? Well, it sort of didn’t. If you’re still looking for that guide, you should probably read “The Flaw of Averages: Why We Underestimate in the Face of Uncertainty” by Sam L. Savage.

I’m not sure it will make your estimates more accurate, but at least you’ll understand why.

 @TSigberg

 

 

 

Advertisements

Posted on July 9, 2014, in Management, Software development and tagged , , , . Bookmark the permalink. 5 Comments.

  1. Yes, estimating is hard, but many ways to assure those estimates are credible. Reference Class Forecasting, Parametric model, tools, paper, books, conference proceedings all show how to do it. As well governance frameworks. For example two the we use for software intensive progams http://www.gao.gov/new.items/d093sp.pdf and http://www.nasa.gov/pdf/263676main_2008-NASA-Cost-Handbook-FINAL_v6.pdf

    The key is to move beyond reporting the symptoms of bad management, look for the Root Causes, and fix them. This starts with “learning” to avoid the symptoms, and install a process that creates credible estimates for the proper “value at risk” effort. Low risk, acceptable “value at risk,” back of napkin estimate is fine. Spending Millions on a High Risk ERP deployment, you’re going to want to have a credible, risk adjusted, management reserve estimate and manage to those estimates with the agile approach of producing tangible measurable outcomes on short term boundaries to update the estimates from the measure of Physical Percent Complete against Plan.

    Lot’s of research and answers to how to avoid the errors mentioned in the book. Just a tiny sample http://cisjournal.org/archive/vol2no1/vol2no1_3.pdf

    If you have a baseline “target” productivity plan, measuring actuals doesn’t provide the needed feedback to meet the business needs of producing value for the budget and on the need date. A working product that is late and over budget has reduced value, possibly zero value

  2. Galleman, thank you for your input on the subject. It’s always good to be able to present and share alternative views. 🙂

    The problem isn’t always that it is impossible to present an estimate, but that it may be hard to communicate that there are a number of possible outcomes (sometimes with quite a wide spread), and not one fixed number. There are ways to reduce the spread, but that also narrows down the available options during the project. Doing that is not always the best solution for the long term result of the project.

    • All etimates are probabilistic, with a Probability Distribution Function, showing the probaility of all the values that or possible. These could be discrete (finite number) or continous depending on the process to create them. So the number of possible outcomes are the basis of decision making.
      The paradigm is the basis of microeconomics. http://goo.gl/wVBfC1
      If we accept this paradigm, and there are some in the #NoEstimates community that don’t then estimates inform the decision making process. With estimats, assessing the impact of the choice to pick on option can’t be informed by the Opportunity Cost or the Selection cost.
      In other words we’re choosing without knowing the cost, schedule, and technical impact in our project domain.

  1. Pingback: Five Blogs – 14 July 2014 | 5blogs

  2. Pingback: Exactly HOW likely is the most likely outcome? | The art of lean management

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: