Big project, big delay?

It all should have been so different. After nine years of construction work on Crossrail, 9 December 2018 was supposed to mark the beginning of passenger services on the most important section of the new Elizabeth Line railway, the underground tunnel linking Paddington in the west of London with Abbey Wood in the east.

However, as is clearly evident from the scaffolded exteriors of many of the stations, this did not happen. The opening has now been pushed back to either late 2020 or early 2021, much to the frustration of London Mayor Sadiq Khan, who said he was “frustrated, disappointed and angry” about the delay.

This shouldn’t, however, have been a surprise as Crossrail is only the latest in a long list of ‘megaprojects’ around the world that have ended up being delivered late and over budget.

For example, Berlin Brandenburg Airport was originally supposed to open in 2011, but now planners expect the first passenger flights to take off in 2021 – a full decade later and billions of euros more than planned. Similarly, the Scottish Parliament building was originally projected to cost £40m when it was approved – but not only was the opening delayed, but costs spiralled to £400m – ten times the original estimate.

In fact, according to the Oxford Global Projects Database, which contains data on almost 12,000 different projects, almost every category run over schedule and over budget: 39 per cent of rail projects are late and 38 per cent are over budget. Similarly, aerospace projects on average cost 61 per cent more than expected and 27 per cent are late .

Perhaps the most extreme category is Olympic Games: They are never delayed, for obvious reasons – but as a result of the immense time pressure, they on average cost 172 per cent what was originally forecast. It perhaps shouldn’t be surprising that Mayor of Rome Virginia Raggi cancelled her city’s bid for the 2024 Olympics, saying: “The Olympics are a dream that turn into a nightmare.”

Perhaps the best way to visualise the scale of the problem is with one example that is popular in the project data analytics community.

Crossrail was originally budgeted with £3bn allocated for contingency purposes – and as the project went on it required an additional £3.2bn for risk and contingencies. Together, this £6.2bn makes up around 35 per cent of Crossrail’s £17.6bn budget.

The UK’s next big infrastructure project – which has already started – is High Speed 2, the new rail line linking London’s Euston station with Birmingham in its first phase. It has a budget of an eye-watering £57bn. If you scale up the proportion of money spent on contingencies and risks that’s another £20bn. In other words, if it were possible to forecast and budget HS2 accurately from the beginning, there could conceivably be enough money left over to build another entire Crossrail.

This raises an obvious question: given that we know megaprojects keep failing in this way, why does it keep happening? And is there anything that can be done about it?

“Over budget, over time, under benefits, over and over again”: this is what Professor Bent Flyvbjerg, who co-created the Oxford Major Projects database, calls the “iron law” of megaproject management.

In other words, imagine a triangle, with each point representing ‘on time’, ‘on budget’ and ‘on benefit’ (ie: does it provide the expected benefits) – it’s virtually impossible to get all three. Of the projects listed in the database, reportedly only 47.5 per cent are recorded as being on or below budget. This falls to 7.8 per cent for projects that were delivered both on time and on budget or better. And the most jaw-dropping statistic? Just 0.5 per cent of projects meet the three pillars of the triangle.

The problems begin with forecasting methodology – which makes sense given how absurdly complicated it is to build something like Crossrail. How does a project manager start?

According to Martin Paver, who has managed billion-pound projects in the nuclear and defence industries, the first step is the obvious one: break down the work into components and then break down the components further, estimating how long each part will take.

Then there’s the risk assessment, the most common form of which is known as a Monte Carlo Analysis. Developed by nuclear scientists in the Second World War, the idea is to randomly allocate a probability to each risk identified in each component, and then run the simulation thousands of times (so that, for example, on one occasion the tunnel might not be dug quickly enough, and in another the electric system needs completely redesigning, and so on). From this, project managers can then derive what’s called a “three-point estimate”, which will give a range of dates – say, the early date where there is a 10 per cent chance the project will be completed, the mid-date where there is a 50 per cent chance of completion, and a late date where there is a 90 per cent chance of the project being done.

However, this sort of approach is inherently flawed, says Flyvbjerg. “We all have cognitive biases, and those biases will influence the forecasts that we do,” he explains, pointing to the Nobel Prize-winning work of behavioural scientists Daniel Kahneman and Amos Tversky.

In other words, humans are the problem, and in a paper titled ‘What You Should Know About Megaprojects’, he points a finger at some of the major biases at work.

For example, projects often suffer from a uniqueness bias: because megaprojects are so large and complicated, it is easy to mistakenly assume that the project is unique; no one else has tried to dig a tunnel under London, which has a number of unique characteristics – so engineers may not think they can learn from, say, the Paris Metro extension or the Boston Big Dig.

Similarly, projects do not plan for ‘black swan’ events – events that are extremely rare but potentially devastating. But given that megaprojects often take years to build and involve such complexity, rare events become likely to happen. In Crossrail’s case, an example of this might have been the explosion of an electrical transformer at Pudding Mill Lane, which not only caused physical damage but also impacted the plans for testing the line’s new trains.

And most of the time with big projects there is an optimism bias – an assumption that something can be built quickly or cheaply. Often with megaprojects, the problem is that the optimism bias affects every stakeholder, as it is in the interests of everyone involved to believe the most optimistic predictions, from the politician who wants to cut the ribbon before they leave office, to the construction and engineering companies that want to appear cheap enough to win the contract.

Similarly, the biases around projects can often manifest as what Flyvbjerg calls “perverse incentives”, with one example being early warning systems and how projects are governed.

“We find that this is one of the key things once construction starts; when things go wrong if there’s no system in place to flag problems to the right people. So it might well be recorded somewhere in the organisation that things are not going well on a part of the project, but there are no rules about how to escalate it,” he explains.

“Instead of ensuring that you’re getting a project that is on time and on budget and delivered, the same incentives are actually set up in a way where you get the exact opposite.”

One stark example of perverse incentives might be the paradox of setting the timetable itself. “Schedules are a double-edged sword,” explains Simon Bennett, a civil engineer and communications professional who was previously head of stakeholder management at Crossrail. “You have to have one to drive delivery, but once you set a date for big projects and publicise it, missing it becomes so damaging that people are unwilling to revise it when they probably should.”

So what’s a better approach? In Flyvbjerg’s view it is to use “reference class forecasting”. This is a technique based on the work of behavioural scientists Daniel Kahneman and Amos Tversky.

“If you want realistic forecasts, you need to de-bias forecast,” Flyvbjerg says.

In other words, rather than base assumptions on the views of experts asking hypothetical questions, forecasts are going to be more accurate if they are based on previous case studies and compiled by people with previous experience.

This accords with Martin Paver’s view. The problem, as he sees it, is that the construction industry is not good at learning from past experience. “What we’re doing is we’re trivialising all of this complexity and project management into three ‘Janet and John’ lessons learned,” he says, alluding to the 1950s children’s reading scheme.

Instead of big, abstract lessons, he believes that it would be more effective to take a large dataset of construction data and apply machine-learning techniques to find relevant information to make reference forecasts.

He gives the example of the challenge of installing an escalator on an underground transport system, something that has been done many times before.

“The first one took 16 weeks, for the second one 15 weeks. And it was the third that took us 27 weeks – and it took us 27 weeks because of these things here that we didn’t account for. We didn’t realise that, we didn’t realise this. And we know that. So when you do the survey, make sure you check for this, make sure you check for that. And then we should be able to put an escalator in between 14 and 16 weeks.”

It’s a great idea in principle but the real challenge, if this approach is to be adopted, is getting hold of the historic data, and for this, Paver has proposed the creation of what he calls a “data trust”.

“You take your mistakes, your compensation events, your inspection data, every bit of data we can find,” he explains, “and we put each bit inside of a little safe.” The “safe” he refers to is a graph database. This database can then persist beyond the completion of the individual project, so that future projects can perform data analytics and learn directly from real-world case studies.

The big challenge of this approach is also one of incentives. Though Paver has passionately pursued a data-driven approach and has got the Oil & Gas Technology Centre on board as his first partner, other companies and organisations have expressed some scepticism, because the data requested is viewed as commercially sensitive. To mitigate this, Paver is proposing a federated ownership and access structure that will only give access to third-party analytics firms, rather than competitors.

It is still early days for the data trust scheme, and persuading the industry to change is a difficult task, but the benefit of fixing the problem is self-evident: there will be more cash to spread around.

There is one final reason why projects might so often break the budget and timetable sides of the iron triangle: the general public and the politicians simply might forget the lessons that should have been learned, as Bennet explains. “Just like the Jubilee and Victoria lines, once open the Elizabeth Line will quickly become a vital part of the network that no-one could imagine not having, and the delays and cost increases will be forgotten by passengers.”

Let’s just hope that the industry doesn’t do the same.