From the book organisational-abilities
< previous page (Applying Approach to Context)
In business, we can’t predict the future - but that doesn’t stop everyone trying.
There are several important reasons why management need to forecast when options will be carried out and outcomes achieved.
Whilst there is no getting away from the need, it’s critical to maintain an understanding that a forecast is a snapshot in time. What we’re working on may become irrelevant. It may need to change. Forecasting should be completely separate from target setting. It should be the act of providing information and probabilities to support decisions. If a forecast becomes a commitment it means that management have misunderstood the term, have integrated it with target setting, and have lost focus on the outcome in favour of the option being worked on. Use a forecast to decide, not to chastise.
A forecast is what we expect to happen given what we know right now. Its most useful when it’s provided as a range and with a probability. A probabilistic forecast. Think of a weather forecast - knowing the probability of rain helps you prepare. This is as opposed to a deterministic forecast which simply points to an exact number. The more ‘known unknowns’ and ‘unknown unknowns’, the less accurate a forecast will be by definition - there’s too much left out of it. If you’re working in a Simple context, then clearly you can follow a plan, along a timeline, and have a good idea of when something will be delivered - and even the likely result - a deterministic forecast would be OK. Even this, though, could be scuppered if say, a global pandemic breaks out during the delivery. When the context becomes more Complicated, it can still follow a plan, however there will be more moving parts and therefore a greater chance of unexpected problems. In Complex contexts the future is highly uncertain and we’re being a lot more honest with ourselves if we start using ranges and probabilities.
If you’re working off a plan to deliver something you’ve delivered many times over, then you can provide an estimate of how long it will take based on the past experience. It took us 2 months to build X, so with the same conditions it’d take us 2 months to build another one. In other circumstances you’ll need to use data on past performance in delivery as a predictor of future delivery. This is your input. It’s crucial for management to be aware that past results can provide an indicator of future performance, but there are no guarantees that the future will play out in the same way. Unforeseen events in the business environment, the global economy and the climate can render a forecast completely irrelevant. Nassim Taleb has written extensively with regards to what he called “Black Swan” events. If you were to use past performance of the economy as a predictor of the future performance, then in 2008 you would have been caught out. Delivery performance, at least in terms of past output can and should be used to forecast future output (because it’s all we’ve got). But always be open to the possibility that the future may bring something that renders previous data irrelevant. That’s why we need to build in contingency, buffers and spare capacity. This is about building resilience. Efficiency is good up to a point, however it makes you susceptible to shocks relating to events that fall outside of your expectations. Taleb deals with the effects of efficiency causing fragility his subsequent book, Antifragile[1]. Optimising for efficiency removes the ability to absorb shocks. Contrast the human body’s capacity to adapt, heal and often benefit from shocks (eg. fasting), with traffic systems that are designed to a certain threshold and buckle when capacity over and above is needed. If you operate the M25 motorway at capacity, it goes no-where. Both the smooth flow of work and resistance require spare capacity; the relationship to forecasting of delivery is that the more optimised an organisation is for efficiency, the more fragile our operations and forecasts are to unforeseen events.
In delivery our forecasts usually look to say when an option will be complete (that is, an output created, or an action carried out). Below you can see some of the typical ways that forecasting is done in delivery.
Here, work items are given time estimates by people that know exactly how long the job takes. They’ve done it, it’s a known quantity and it’s suitable for traditional waterfall project management.
Forecast = Critical path
This approach only works if the estimates are accurate. For this to be true:
Most work in large organisations will sit outside of these criteria. Even if work has been done before, the way it integrates with other work and with other teams will often be untested. That’s why most large projects or initiatives are destined to be late and over-budget. In fact, the bigger the project the more likely it is to be over budget due to the increasing complexity.
The following forms of forecasting are based on business cadence, that is, using proxies for quantities of past work to forecast when another quantity will be done.
We’ve already discussed the fact that Management shouldn’t use Story Points as a measure of progress since they’re too far removed from verifiable value. The more complex the context, the more unlikely it is that teams will consistently get the sizing in the right ballpark, the less reliable the forecast. But when they do achieve a good level of consistency and get the measure of how much they can do within a time period (velocity) then they can use the information to forecast when future work might be completed.
Deterministic forecast = (Remaining number of story points/Average Velocity)
This kind of forecast will provide a single number - whether of sprints or a delivery date. This can quite easily be misinterpreted by management as a commitment. There’s no indication about how probable it is that it’ll be achieved.
As we’ve discussed, since we should have delivery data, a better way to forecast is to provide a range and a probability. What’s the point of showing the deterministic forecast? I’ve done this because in some situations it may be appropriate; you may need some ‘back of the napkin’ calculations done in a hurry. You may see others doing it. It’s useful to know the difference.
To create a probabilistic forecast, you perform what’s known as a Monte Carlo simulation. Plugging in the previous velocities, the simulation will run 10,000 tests based on the data we’ve input and provide a histogram of all its results. This gives us our possibilities and the relevant probabilities. As an example, now instead of giving one date, we can say that there’s an 80 percent chance that the option will be finished by mid- July. Not only does this provide richer data, it also clearly implies that this is not a certainty. Whilst you may wish you could have an exact date, knowing the reality is much more helpful in decision making. Plenty of exact dates have been given, costly preparations made, and the dates missed. Acknowledge the uncertainty and plan for it.
This approach to forecasting hinges on the Story Points being a proxy for the lead time on work. In environments where the lead time is affected by handovers and significant wait time it will not produce a forecast worth using. How you fix this is by creating teams that can deliver end to end, which will provide greater consistency and predictability.
This type of forecasting uses the number of items completed over a given time period (throughput). Many organisations use User Stories. I class them as a type of option. No matter what you call them, as long as there’s a relatively even distribution of sizes, throughput does provide a relatively good way to forecast future work. Again, we can forecast in this way both deterministically and probabilistically.
Deterministic forecast = (Number of stories remaining / Throughput)>
For the probabilistic forecast again you plug in the throughput data gathered over a time period and run the Monte Carlo simulation. The histogram will provide the range and probabilities.
Any type of forecast is based on past data or information. If inaccurate data is used, or the conditions change, then the forecast is likely to be much less accurate.
Finally, I will leave you with a quote from organisational theorist Russel Ackoff:
“A good deal of the corporate planning I have observed is like a ritual rain dance; it has no effect on the weather that follows, but those who engage in it think it does. Moreover, it seems to me that much of the advice and instruction related to corporate planning is directed at improving the dancing, not the weather.”[3]
To use forecasts, set up a stable environment and system which will allow for some level of consistency. In doing so you will gain a much greater level of predictability. However create resilience in your organisation to ensure that any shocks are absorbed when they come.
The Good - more of this
The Bad - can be misused or produce negative results
The Ugly - bad practice in most organisational contexts
[1] Taleb, Nassim Nicholas, and Nassim Nicholas Taleb. Antifragile: Things That Gain from Disorder. Incerto, Nassim Nicholas Taleb[…]. London: Penguin Books, 2013.
[2] Velocity is a term used for average number of story points “banked” by completing features over a regular time boxed cadence.
[3] Ackoff, Russell Lincoln. Creating the Corporate Future: Plan or Be Planned For. New York: Wiley, 1981. p. ix