It’s time to let you in on The Secret. Don’t tell anyone, but “How long does it take” is the wrong question.
I planned for this post to be about statistics and how to calculate probabilites. Instead it turned into a rant, of sorts. You see, there’s another Secret buried here: If you really take the time to learn how to estimate risk correctly, this will help you learn the following fact about the outcome of your project: It is very uncertain. I guess that’s helpful. Sort of.
“That’s just baloney,” I can hear some of you say. “My projects are consistently on time and on budget,” you continue. Maybe so. Let’s examine the famous “project triangle” for a moment.
Ever wondered why it isn’t a square? Because quality is a result of the other three, some might say. “That’s just baloney,” you can hear me say. I think it’s because cost, scope and time is easy to measure and easy to adjust. Quality? We always deliver perfect quality! We are professionals! Let’s just put that in the middle, and hope no one pays too much attention.
You, the guy who screamed “We always deliver on time and on budget!” a moment ago. How about scope? “Hah! We delivered all the functionality as well! All the developers bitched about them having too much to do in too little time, but my project plan showed them wrong!”
That’s impressive. But even when we go to the painstaking length of figuring out the risk and uncertainty of a project PROPERLY – You know, with Statistics and stuff, not just, God forbid, guessing or anything – it still comes out as being very uncertain. So how come you can consistently deliver your projects on time, scope and budget? Your team don’t have much choice, do they? They have to reduce quality. Luckily for you, no one will notice until the project is over. And they couldn’t really measure it if they did. What IS quality anyway? Sounds like a made up word to me.
So what did I mean by “how long does it take” being the wrong question?
There are a couple of problems. First of all, what is this “it” that we are doing? When you start the project, odds are neither you or the customer knows what the end result is supposed to look like. You may think that you do, but you don’t.
If it is a waterfall project, the customer will say “This is what I want” while dropping the dreaded Complete Requirement Specification on your desk. Then you go away for three months with a team of developers to create whatever is in the specification. “Here is your product!” you exclaim with great enthusiasm when you come back. After the customer has tested it, she goes “Nope, that’s not it.”
If it’s an agile project, your team of developers only go away for maybe a couple of weeks at the time. Each time you come back and show a little bit of the product, the customer tests it and goes “Nope, that’s not it either. And where’s the rest?”
However you go about it, both you and the customer learn a lot DURING the project. Mostly what they don’t want. Needless to say, this process takes quite a bit longer than just doing stuff once. No one knows what done looks like until, well, you’re done. And please remember, we’re not BUILDING a product, we’re CREATING it. The first of its kind. Your developers aren’t packing meat in boxes along a conveyer belt, they’re freaking SCIENTISTS! So stop pretending you are a production facility. The product you’re creating has never been built before. If it has, please go and buy that instead.
And that’s not even half of it. What did you say were doing again? A project, right? Well, most of the time we’re not really creating projects, we’re creating PRODUCTS. The funny thing about products is that they don’t end when the project ends. That’s kind of when they start. In terms of cost, that means roughly 90% of the total cost of a product comes AFTER the initial release. Maintenance, bug fixing, additional development, paying back technical debt, people spending most of their day swearing over how horrible it is, and so on.
Here’s the Third Secret: The less time you spend on quality during the “project” phase, the higher the total cost of the product.
So whenever you pat yourself on the back for delivering “on time” (usually a random point in time decided by anything except the actual amount of work that needs to be done before that date), that probably means you have reduced the overall quality of the project, and increased the total cost of the product you delivered. That doesn’t really sound like back patting performance, if you ask me.
Did you think this series of posts would culiminate in How To Estimate Accurately In 3 Easy Steps? Well, it sort of didn’t. If you’re still looking for that guide, you should probably read “The Flaw of Averages: Why We Underestimate in the Face of Uncertainty” by Sam L. Savage.
I’m not sure it will make your estimates more accurate, but at least you’ll understand why.
(This is part #2 in my mini series of blog posts about estimates. Part #1 can be found here)
With the help of detailed analysis, an experienced developer may figure out the most likely outcome of a project. But exactly HOW likely is the most likely outcome?
In the previous post we were forced to guess the duration of a project, and ended up guessing the project would take 300 hours. As a side note, the result after detailed analysis will probably be quite close to the same number. This is known as anchoring. In order to keep each post relatively short, we will save that for another post.
Let’s assume you’re given time to do a detailed analysis of the proposed project, consider the risk and are provided with a “complete specification”. You spend maybe half a day thinking about it, jot down some detailed estimates, and add them all together. The total sum is 290 hours, and you figure that to be the most likely outcome of the project. Your assumption may be exactly wrong for any number of reasons, but for the sake of argument, let’s assume you’re actually correct.
You have now figured out that the project will most likely take around 290 hours. But exactly HOW likely is it that the estimate matches the actual end result? 60%? 80%? You may be surprised to learn that the actual number is probably quite a bit lower than that.
Above illustration represents the actual outcome of 42 Norwegian projects from several different companies. (Magne Jørgensen / scienta.no)
Let’s say you have historical data from several past projects, telling you both your estimated, most likely outcome, as well as the actual outcome. If you put them in a graph with the x-axis representing the actual effort in percent of the estimated effort, and the y-axis representing the percentage of the total number of projects, it would probably look something like the graph above. Confused yet? Let me explain that again.
The highest bar in the above graph is the one marked “100” on the x-axis, and the value of that bar is around 33. That means that roughly 33% of the projects were completed on the estimated time. Since it’s the highest bar, being on time appears to be the most likely outcome. The next bar to the right, marked “125” on the x-axis, indicates that a little over 15% of the projects actually spent 125% of the estimated time, and so on.
So why do you care? Because it sheds light on what “most likely” really means. And it may not be what you think. Actually, our most likely outcome (hitting the estimate spot on) is NOT very likely to happen. Yes, it surely is more likely than any other individual result, but it is LESS likely to happen than all the other results combined. We actually have a whopping 67% likelyhood of NOT hitting our estimate, even though (due to some unknown miracle) our estimate is the most likely result of all possible results.
To make matters worse, the combined likelyhood of the bars to the LEFT of the 100% bar (representing actual outcomes that were lower than our estimate) is only about 7%. So we got 33% likelyhood of hitting our estimate, and 7% likelyhood of being below.
If you provide your boss with your “most likely” estimate, you will in this example leave him with a 60% chance of blowing the customers budget.
I’ll leave you with some time to think that through. Part #3 of this mini series can be found here.
The middle manager. The useless fat of any bloated organization. They delegate all their work to other people, and then they wander around aimlessy, doing nothing at all expect worry about what happens if the delegated work doesn’t get done in time.
Being useless fat can be both rewarding and fun, but most of the time it is a difficult job. Even so, it must be done. After all, stuff doesn’t get delegated by itself..
It’s an interesting and not entirely new question. What to do with middle management in a company that is agile, stuffed with self organizing teams that in turn are made up of super-cool and smart people that don’t really care very much for authority figures who try to tell them what to do. Do we really need them around?
In an “agile organization” (whatever that is), stuff still needs delegating. But perhaps you are delegating goals and projects, instead of tasks. Less time is spent reporting against imaginary project plans, more time is spent actually creating value for the customer. Sharing information, explaining goals. Discussing possible solutions. Developing said solutions, showing them to the customer early and often.
As an agile or lean manager, I’m not really the boss of someone else. I am simply responsible for other parts of the value delivering process. I am also responsible for optimizing the process itself. I also try to inspire everyone else to optimize their part of the process, whenever that is possible without hurting the process as a whole.
My job is to protect tech people from nosy sales people, help sales people understand difficult tech people, explain to owners how a little money now isn’t always better than a lot of money later, and that more quality actually equals less cost in the long run. If all of the above goes well, the result is surprisingly often that we deliver valuable and useful software to customers who don’t always know what they need, but always know where it hurts.
Basically it is understanding, sharing and aligning goals and mindsets. And that is a really fancy way of saying that you need to talk a lot. I don’t really like talking to people very much, so I guess that explains why I spend half the time worrying instead.
It may not be very effective, but at least worried people look busy. And looking busy is important when you are useless fat.
The main problem with sharing information, is that most of the time people just don’t listen very well.
“What do you mean the project is late?!”
“Boss, It was always going to be late. It was doomed from the start.”
“So why the hell didn’t you say so?”
“Actually, I did.”
Being a software developer isn’t easy. Working on projects that don’t deliver as expected are common. However, most software projects don’t end up late unexpectedly – they were bound to be late before they even started.
Let me share a few typical reasons:
- The project manager allocates 8 hours of your time to the project each day, even though he knows all too well that you spend 2 and sometimes even 3 hours per day handling support, production issues and answering questions from management and sales. Some days you even have lunch, even though it’s against company policy. Now you’re 20-30% behind schedule already. Swell!
- Someone set a deadline before anyone have a clue what to solve, and much less how. In reality, most deadlines don’t even signify an important event for the project or product. It’s just an artifical date set because the customer decided “It has to be done by December 1st!”. No, it really doesn’t.
- You’re only allowed to spend an absolute minimum amount of time analyzing customer needs. And by all means, don’t attempt to figure out the best way to solve the problem. “The deadline is set already, we don’t have time to sit around thinking and wiggling our toes! Start DOING something, for heavens sake!”
- Management ignores any concern or warning voiced by the development team, and the team accepts any assignment, however unreasonable. “But boss, this is never going to be done on time!” “Well, it has to be! You’ll just have to find a way!” “Oh, okay. I guess we’ll have to figure out a way to deliver everything you’ve promised on our behalf without reducing quality or functionality. I mean after all, we’re basically wizards over here.” No, I’m afraid you’re not.
Bosses and sales people insist that you try to deliver, however unrealistic the goal may be. “We can’t let the customer down!” In the long run, giving in to unrealistic expectations actually hurts the customer. Each time the development team takes on more than it can handle, it compromises either quality or time. More time spent on one customer, means less time spent on another. Delivering bad quality and bugs, means even less predictable availability later on.
In order to satisfy all our customers in the long run, we need to be as effective as we possibly can, and deliver consistent quality at a pace that the team can handle over time. That may mean some difficult discussions and difficult decisions. However, it is better to adjust the expectations of the customer now, than to disappoint him with an inferior product or a missed deadline later.
W. Edwards Deming (Wikipedia)
The system. W. Edwards Deming claimed that 95% of the performance of an organisation is based on the system, and only 5% is based on the people. I won’t dare go so far as to contradict Mr. Deming, but as a manager in a small company, I am tempted to look at it from a slightly different angle.
A brilliant employee will never perform in a mediocre system, and a mediocre employee will do a decent job in a brilliant system. But if your company is twenty, ten or maybe just five guys (or girls) – surely, people must matter?
You can not blame the worker for wrongdoing, most defects and quality problems are a result of the system. But if you are five guys sitting in the same room trying to make your company succeed – well, you are the system. The five of you. When Joe screws up and says to the other four guys, “Sorry, I’m a victim of the system!” they will probably look around the room and go “Uh, what system?”
But wait a minute. However small a company, there are someone in management, there are someone in sales, you have a customer department, and you’ve (hopefully) got someone actually producing whatever it is you are attempting to sell. Maybe Joe is both management, sales and customer department, but those roles still exist in the company. So however small, your company is an ecosystem. And if you haven’t given that much thought, possibly not a very good one.
You may just be ten guys, or just five. That doesn’t really matter. You should already be thinking structure. Processes. Quality. Why didn’t Joe tell Pete about the complaint from their most important customer? Why did Pete forget to test the new software that Mike just hacked together? And why did he hack it together when they all agreed last month that the software needed to be robust and well built?
I can hear you shout: “We can’t afford all that stuff, we’re just a small startup company!” Well, to paraphrase old William Edwards Deming: If you focus on quality, quality increase and costs fall over time. However, if you focus on cost, costs will rise and quality decline over time. Surely, that’s not what you want?
So did I answer my own question? Well, kind of. Even though “people are only 5% of the performance”, in those early days you need people who understand just that. That takes us back to where we started, perhaps having gained somewhat of a paradox along the way:
People are helpless to fight a faulty system, but to build the right system, you need the right people.
Customers who are unable to explain what they really need, marketing guys that make up their own stories, and project owners that won’t take the time to give you feedback during the project. A recipe for disaster.
I’ve recently discovered Prezi, an online and fun alternative to powerpoint. So I decided to make a little prezi-based blog post to test it out. Enjoy!
A late project? Well, it has been known to happen. According to my wife (who has got a far better memory than me), almost all my projects are late.
I attempt to explain that they are not really late, someone has just decided to add more features. Or we all decided to postpone the release, to make sure the delivery had the required quality. Then she goes “But the project was supposed to be done last tuesday, right?” and I’m like “Yes, but..” and she cuts me off “So that makes it late.”
A popular term is “definition of done“. Maybe we should also discuss another: “Definition of late”. The definition of done may be challenging to define, and across an organization it’s very likely to mean different things to different people. One would think that definition of late should be easier to define.
“The project should be done by November 1st. If it isn’t, then it’s late!” Right. But wait. Done? If the project isn’t done until September 1st it’s late. Suddenly avoiding being late relies on something being done – and we just agreed that “done” is a tricky topic. Or is it?
When we postpone the release to production in order to do some additional QA, we’re not really late right? We’re done with the project, we just haven’t released it yet. That must be okay? Or does the lack of quality mean that we tried to deliver too much in too little time?
And if the project owner asks you to add more features, surely that’s no problem. He is the boss, after all. I suggest you explain to him that it will postpone the delivery of all the value created so far. A better solution would be to deliver what we agreed, so that the project outcome will benefit the customer as soon as possible. Then we can add more stuff in the next round.
So, back to being done and late. You see, the thing is that the customer rarely considers a project as done before he can create value from the result. So if your customer can’t create the expected value from whatever he ordered from you, then you are not done.
And if the customer can’t create the aforementioned value by the agreed delivery date, then my friend..
You are late.
I’m pretty sure that if a developer built a house, it wouldn’t have any corners. Their first instinct at the sight of an obstacle would be to get rid of them. Corners you say? Who needs them! And then they’ll all start chanting: Cut a corner, cut a corner!
At Revio, we challenge the developers to deliver estimates that they believe in. We tell them not to consider the expectations of management or customers. Consider the task at hand, and attempt to figure out how long it will take to get it done, and done right.
As discussed in an earlier post, accurate estimates are hard. That means even though we estimate with the best intentions, we might still end up hard pressed for time. And even in the flowery, sunny world of our little company, we occasionally misunderstand the customer. So there we are, with maybe 3 developers in the team, ten days until deadline, and 500 hours of work yet to be done.
Fig. to take shortcuts; to save money or effort by finding cheaper or easier ways to do something.
To do something in the easiest, quickest, or cheapest way, often harming the quality of your work.
In a situation like this, the vision of an unexperienced developer often narrows down to the very next line of code as he mentally calculates how to shortcut his way out of his misery. Must. Code. Faster.
Let me share with you what I as a manager would want the developers to do in this situation:
- Give me a calculator, help me type in 3 developers times 8 hours times 10 days equals 240 hours. Explain that we have 500 hours yet to be done. Then ask me patiently what I would like to do with the 260 hours worth of work that we do not have the capacity to complete.
- Tell me to read “The mythical man month” if I suggest to add another 3 developers to solve the crisis.
- Ask me wether moving the deadline is a viable option (More often than not, it actually is).
- Work together with me in order to figure what features can be moved to a later phase, while still delivering as much value as possible in the current phase.
- NEVER EVER cut on code quality (architecture, coding practices, coding conventions).
- NEVER EVER cut on quality assurance and testing.
If you discover mid-project that you have more work than you have hours available, you will not get everything done in time. Period. Going into a team-wide panic and start building up technical debt like crazy won’t really help. In the long run, it’ll just make things worse.
Often, quick and dirty actually takes even longer. This is especially true if you have poor specifications and need to do a lot of re-work. When you tell me “yes we can” when actually we don’t, that’s bad for both of us.
So in conclusion: The next time you’re stuck in a doomed project, make sure you stay realistic and maintain your own integrity. Your manager may ask you to get 20 days worth of development done in 10 days, but it’s up to you to accept it.
The guys in the agile community have been writing, talking and moaning about estimates for ages. “Accurate estimates are impossible! What’s the point? Let’s stop estimating, it’s just guessing anyway!“
Let me try to explain.
We all agree that we’re basically guessing, right? That may be the case, but if you live in the real world, you actually sell stuff for a living. In order to sell stuff, you need to have customers. And what’s the problem with customers? They usually demand to know what something costs before it’s produced. Go figure. Anyway, that means we need to estimate wether we like it or not.
A rough estimate from a developer or architect is usually the basis for a fixed price offer. If the estimate is good, you’ll make money. Sometimes it’s not, and then you’re up shit creek. If bad luck and an inexperienced developer team up, rough estimates can be several hundred percent off the mark. So why do we keep working like this?
The big problem with this transaction is trust. The customer rarely trust the software company enough to pay for a project by the hour. The history books are full of dishonest companies who have destroyed the trust in the IT industry. Now we all have to work hard in order to regain that trust. You need to convince each and every customer that you actually want to add value to their business by creating great software. That’s what we’re all about. Right?
If you gain enough trust within your market, I strongly suggest that you attempt to operate with budget limits instead of a fixed price. Agree on a minimum scope, and explain to the customer that you will do your best to include as much functionality as possible within the budget you have agreed upon. This way you reduce the risk on your end, and the customer doesn’t have to pay for the built-in risk premium of a fixed price. A better deal for everyone.
Great stuff, happy campers all around! Now we can stop estimating and focus on real work.. But wait a minute. How can you (or the customer) know wether the project will be worthwhile? How do you establish a reasonable budget? When presented with the required time to market, how can you tell if you will be able to deliver on time? You guessed it, by estimating. Seems like there’s no way to win this game.
To make matters worse, the game changes as you go along. As your company grows, your development process will change. You add quality assurance, you add a project manager. Your team grows, and suddenly lots of time is spent talking about how to do stuff instead of actually doing it. Challenging SLAs means more time spent on release management and server maintenance. The environment surrounding your team changes all the time, and the correct answer to “how long will it take to..” changes with it.
At the end of the day, an estimate can never be more than an educated guess. But fear not, all is not lost: If you manage to build a team with focus on quality, keep the turnover low and maintain a stable, honest environment with room for making mistakes – you might just end up with a bunch of guys that are amazingly good at guessing.
I’ve got a list of what I try to focus on as a CTO and project manager. On my list I’ve got 7 items. Project management is number 5.
Project management. Some dork in a tie running around while he’s pointing at his gant chart and yelling at people to work faster. A gant chart that stopped resembling reality before it’s even printed. He has never delivered a project on time. Ever. Everyone knows projects are always late. There’s always some delay or scope creep or useless third party vendor with a product that doesn’t work. So why even bother?
But wait.. Does another breed of management exist? Living in a touchy feely world, where projects actually deliver value, on time? And if they had a list, would it look anything like this?
#1: Be there for your project team:
Care about their problems, and help them resolve them. Your role is to remove any obstacle, and for the love of God, make sure you’re not an obstacle yourself. When they’re working, stay out of their way.
#2: Quality assurance:
Most project managers (myself included) spend too little time focusing on what is actually being done. Are your project output in tune with what your customer expects, and does it create actual value? Cost. Scope. Time. There are always someone trying to make you do more for less. Tell them to stick their project triangle where the sun doesn’t shine, and replace it with Quality, Quality, Quality.
Effective projects depend on good processes. Strive to be better. Lock into a lean mindset where you continously attempt to improve the way you and your team work, and eliminate waste.
#4: Customer service (and zero tolerance for incidents that negatively impact the customer):
You must always be available to your customer. You need make sure they know what to expect, and when they can expect it. Oh, and the last part of this rule I stole from my previous boss:
Me: “We’ve got a bug in production!” My boss: “Does it affect our customers? Forget what you’re doing and fix it!” Me: “Now?” My boss: “Are you still here?”
#5: Project management:
Is everyone and everything on track? Has important milestones been met? Do all stakeholders have the information they need? You know the drill.
#6: Sales and marketing support:
Don’t forget tomorrow. Another day, another project. Your support to the marketing division will help them make the right decisions based on your input. This ensures that the next project they’ve got cooking gets a reality check, and thereby is granted at least a moderate chance of success.
A gant chart is a great visual aid to help the customer understand when you will deliver project output. As a management aid it is more or less useless. Plan 1-2 weeks in moderate detail (not in gant chart!), 2-3 months as a rough draft to know how many resources are booked and how many (if any) will be available for additional projects. That’s it.
So, there you have it. Is this a perfect list? Far from it. Do I even follow it? Not always.
But no matter how busy my day gets, I try to keep at least #1 and #3 on my forehead. And the rest in my back pocket, for safe keeping.
How about you?