Exactly HOW likely is the most likely outcome?

(This is part #2 in my mini series of blog posts about estimates. Part #1 can be found here)

With the help of detailed analysis, an experienced developer may figure out the most likely outcome of a project. But exactly HOW likely is the most likely outcome?

In the previous post we were forced to guess the duration of a project, and ended up guessing the project would take 300 hours. As a side note, the result after detailed analysis will probably be quite close to the same number. This is known as anchoring. In order to keep each post relatively short, we will save that for another post.

Let’s assume you’re given time to do a detailed analysis of the proposed project, consider the risk and are provided with a “complete specification”. You spend maybe half a day thinking about it, jot down some detailed estimates, and add them all together. The total sum is 290 hours, and you figure that to be the most likely outcome of the project. Your assumption may be exactly wrong for any number of reasons, but for the sake of argument, let’s assume you’re actually correct.

You have now figured out that the project will most likely take around 290 hours. But exactly HOW likely is it that the estimate matches the actual end result? 60%? 80%? You may be surprised to learn that the actual number is probably quite a bit lower than that. Above illustration represents the actual outcome of 42 Norwegian projects from several different companies. (Magne Jørgensen / scienta.no)

Let’s say you have historical data from several past projects, telling you both your estimated, most likely outcome, as well as the actual outcome. If you put them in a graph with the x-axis representing the actual effort in percent of the estimated effort, and the y-axis representing the percentage of the total number of projects, it would probably look something like the graph above. Confused yet? Let me explain that again.

The highest bar in the above graph is the one marked “100” on the x-axis, and the value of that bar is around 33. That means that roughly 33% of the projects were completed on the estimated time. Since it’s the highest bar, being on time appears to be the most likely outcome. The next bar to the right, marked “125” on the x-axis, indicates that a little over 15% of the projects actually spent 125% of the estimated time, and so on.

So why do you care? Because it sheds light on what “most likely” really means. And it may not be what you think. Actually, our most likely outcome (hitting the estimate spot on) is NOT very likely to happen. Yes, it surely is more likely than any other individual result, but it is LESS likely to happen than all the other results combined. We actually have a whopping 67% likelyhood of NOT hitting our estimate, even though (due to some unknown miracle) our estimate is the most likely result of all possible results.

To make matters worse, the combined likelyhood of the bars to the LEFT of the 100% bar (representing actual outcomes that were lower than our estimate) is only about 7%. So we got 33% likelyhood of hitting our estimate, and 7% likelyhood of being below.

If you provide your boss with your “most likely” estimate, you will in this example leave him with a 60% chance of blowing the customers budget.

I’ll leave you with some time to think that through. Part #3 of this mini series can be found here.

@TSigberg