Software predictability and estimation have always been problematic. (I have some theories about why it’s worse for software than other engineering disciplines but that’s for another post.)
Since the advent of Agile practices, I think predictability and estimation have gotten worse, for a relatively simple reason - Agile (in all of its variants) devalues predictability in favor of iterative and (relative to waterfall) fine-grained deliverables. The resulting mindset tends towards “we don’t know what we’re building in the long run so how could we possibly estimate it?” Or even “It’ll be ready when it’s ready.”
Of course there are tremendous benefits to a more interactive and fine-grained approach. I never want to go back to 2+ year development cycles that I experienced early in my career.
But there is also value to predictability and estimation accuracy. And saying we can only have one or the other is a false dichotomy.
Let’s spend a few minutes enumerating the value of being good at estimating and predicting software development
- Product teams make decisions based in large part on cost-benefit analyses. Both the cost and benefit sides have huge error bars, of course. But we do the best we can. Better estimates lead to better decisions. If we under-scope badly we may start a project only to discover that we would have been better off doing a different project. (Or if we over-scope badly we may leave low-hanging fruit opportunities unharvested.)
- Predictability engenders trust. We are more trustworthy if we make commitments. And we’re even more trustworthy if we meet them. This is true whether we are making commitments to external customers or internal stakeholders. Consider how you would evaluate two vendors - one who makes and meets commitments and one who says “it will be done when it’s done.” Putting the shoe on the other foot, if you want to be a trusted and respected vendor (to internal or external stakeholders) you want to make commitments and meet them.
- The team feels enormously better when it sets expectations and meets them rather than always feeling “late.”
Here are some tactics that help with predictability and estimating accuracy.
- Lack of understanding of a problem and its solution is the primary cause of poor estimation. Deep understanding results in more accurate estimates. I think everyone, even those who are loath to commit to dates, agrees that deep understanding is better than shallow. So aspire to getting better at understanding your problems and solutions and estimates will improve organically.
- There are always unknowns. Tackle these early and aggressively. Use prototyping, mocking and research “spikes” to close the error bars on the largest unknowns. Resist the temptation to focus on the “known knows” first because you know how to do them - the “known knowns” are never the reasons estimates are poor and schedules are unpredictable.
- Don’t overthink it. There are limits to how much we can improve estimates with thought exercises. At some point you just have to start. I rarely quote Aristotle regarding software development 😀, but he observes: “... rest satisfied with the degree of precision which the nature of the subject admits and not ... seek exactness where only an approximation is possible.” Nicomachean Ethics, Book 1, Chapter 3.
And here are some thoughts on how to manage estimates in your larger organization.
- Always (always) quote ranges, not dates. Let the consumer of the estimate decide whether to be optimistic or pessimistic. (Pro tip: do this in writing so there is never ambiguity about what was communicated.)
- “Sand-bagging,” or deliberately and arbitrarily padding estimates is a trust-breaker. Sooner or later sand-bagging will be discovered and it’s extremely difficult to recover trust after that. If you truly don’t think you can estimate, you’re better off saying so than giving an arbitrary estimate.
You’ll be a better software engineer, and a member of better teams, if you can deeply understand your problems and solutions and thereby make confident and reliable estimates.
