Why large-scale public transport projects go off the rails

Existing systems of scrutiny and audit need strengthening to stop high-profile failures like HS2

February 14, 2024
An HS2 construction site at Euston. Image: ZUMA Press, Inc. / Alamy Stock Photo
An HS2 construction site at Euston. Image: ZUMA Press, Inc. / Alamy Stock Photo

The allegations against the Post Office in the Horizon scandal may include a phenomenon observed in a number of public projects: officials who have been given a remit of “delivering” a project feel justified in dissembling because of a concern that being frank would lead to the cancellation of their project or jeopardise their organisation. The deception is not for direct personal gain, but out of loyalty or perceived professional obligation. This may manifest as the obstruction of outside scrutiny, but sometimes it has gone beyond that. There may be an element of an optimistic inability to believe the unfavourable evidence available.

For ventures entirely within private-sector companies, late delivery, disappointing performance and spiralling budgets are at the risk of the owners—often shareholders—and it falls to them to act to protect their interests. But for projects where there is a wider public interest and significant public funding, there has to be accountability to local or national parliaments. These parliaments must make judgements on whether starting or continuing with a project is a proper way of spending public money, and to do that they have to trust those planning and delivering the projects. If those people withhold information or relay inaccuracies in order to increase the chances that “their” projects would receive approval, or to escape cancellation, then good public administration will be compromised. Sadly, there have been documented cases of this in the transport sector, suggesting that the current systems of public audit are not adequate and need to be reviewed. 

Failure of railway timetabling

One example was the failure to implement major national railway timetable changes in May 2018. Just days before the event, ministers questioned the train operators who gave assurances that everything was in order. However, in the event, rail services in two large areas of the country became chaotic. The subsequent investigation by the independent Office of Rail and Road (ORR) recorded that hundreds of trains didn’t run on weekdays during the disruption.

The ORR established that, despite their assurances, some train operators had known they were substantially short of the drivers and equipment needed, and that they were likely to fall considerably short of properly delivering the timetable for several weeks. Subsequently, Network Rail put more resources into its “system operator” function and returned to ensuring industry compliance with the advanced deadlines to declare changes as set out in train operators’ operating licences. Train operators procured the resources necessary to run their services and the problem appeared to have been resolved until the Covid­-19 pandemic, industrial unrest and other resourcing failures caused it to return in 2020.

The Elizabeth Line (Crossrail)

Prior to the summer of 2018, officials responsible for building Crossrail (now the Elizabeth Line) had assured ministers, the mayor of London, the Railway Inspectorate at the ORR and others that the project was within budget and would open in December 2018. However, suspecting that not all was well, Transport for London (TfL) and the mayor investigated, quickly establishing that there had been no hope of opening for passenger service and more work and more funding would be required. The Elizabeth Line did not open for passenger service until May 2022. 

Jacobs consultancy had been appointed in 2009 by the government as Project Representative for Crossrail to “provide oversight support to the project sponsors (Department for Transport and Transport for London) to ensure that Crossrail Ltd will deliver the project on schedule, within budget and to the agreed standard.” Jacobs had been warning of looming difficulties for some time, but it seems was not adequately heeded. Frankness about the realities and earlier action in mitigation would have allowed the line to open earlier and at less cost.


HS2 was developed in an open manner. Rather than keeping it within a government department, HS2 Ltd was set up in 2009 as a separate, government­ owned company. In the early years there were three advisory panels (strategy, technical expert, and analytical “challenge panels”) with independent members. Throughout the period from 2009 to April 2020, when Boris Johnson made the final decision to proceed with the full scheme, HS2 Ltd and the Department for Transport published their own studies and many of the reports by consultancies.

However, there was still a lack of frankness about the developing understanding of the likely engineering costs. Lord Berkley’s 2020 report into HS2 cited a 2016 letter from then-Secretary of State for Transport Patrick McLoughlin to George Osborne, which admits that HS2 could not be kept within budget but insists this must be kept secret from parliament. Yet, as late as July 2019, transport minister Nusrat Ghani reassured parliament that “there is only one budget for HS2, and it is £55.7bn [set in 2015]”. Not many people noticed that this statement refers to the “budget” rather than the cost estimate. 

For some time, the government had been holding the position that the scheme would be delivered to the 2015 budget figure, but only by achieving yet­ to ­be ­identified “efficiency gains”. A later review by the House of Commons Public Accounts Committee concluded: “The Department and HS2 Ltd were aware of the scale of the issues facing the programme as early as October 2018. In March 2019 HS2 Ltd formally notified the Department that it could not deliver Phase One to budget and schedule. Despite being aware of these issues, the Permanent Secretary withheld from us that the programme was in significant difficulty when she appeared before the previous Committee in October 2018 and May 2019… HS2 Ltd’s annual report and accounts for the year ending 31 March 2019 similarly failed to give an accurate account of the programme’s problems… no adequate excuse was provided for not disclosing to this Committee and Parliament the risk and uncertainty the programme was facing… Lack of clarity and obfuscation about the budget issues with HS2 risks jeopardising the trust between Parliamentary committees and Government officials.”

Independent certification of any company’s annual report and accounts is a crucial element of protection of the interests of the owner, private or public. Inadequate external audit is a serious matter. This was illustrated by the Enron case and by the heavy fine imposed on Carillion’s external auditor by the Financial Reporting Council after the company’s collapse. Although some public bodies still use accountancy firms as their external auditors (for instance Transport for London), it is now not unusual for the National Audit Office to certify their annual report and accounts—and this was the case with HS2 Ltd. So, the National Audit Office certified the annual report and accounts which the Public Accounts Committee criticised. But the National Audit Office is also the principal technical advisor to the Public Accounts Committee and it makes periodic reports to parliament on the governance and general performance of public projects. With the benefit of hindsight, the risk of making the National Audit Office’s overall “watchdog” function less effective might have been avoided by using a different organisation as independent, external auditor for HS2 annual report and accounts.

The London Underground Public Private Partnership

On taking office, the 1997 Labour government announced a policy of striking a new spirit of “partnership” between government and private sector providers. In a speech in 1998 paymaster general Geoffrey Robinson said: “I was shocked by the consultation we conducted with industry in Opposition how far mutual mistrust had developed between the public and private sectors to the detriment of the reputation of both and of the effectiveness of the whole [Private Finance Programme] programme.” Later that year, the government explained in response to a select committee report that “The [Public Private Partnership] represents a new approach with public and private sectors working together in partnership. It follows that we wish to craft the contractual relationship in such a way as will enable the majority of any changes to be made by agreement, with any disagreements resolved between the parties.” So the aspiration was that the private sector could be harnessed to deliver public services with less reliance on litigious enforcement of formal contracts and a greater spirit of cooperative partnership.

In the event, the government spent nearly six years creating the London Underground Public Private Partnership (LU PPP), one of the largest and most complex commercial contracts imaginable, and in due course there was major litigation in front of the independent PPP Arbiter. The details of the contracts were negotiated privately, the justification offered being that the government did not want to compromise its commercial position. Parliamentarians complained that they felt that information was being withheld. The mayor of London and his Commissioner, who would be saddled with the signed contracts, were largely excluded from the negotiations. Treasury ministers and officials declined to appear in front of the parliamentary Transport Select Committee, which is “unusual” under the constitutional conventions. This left parliament unsighted about the nature and magnitudes of the contractual commitments to spending public money—which changed a great deal during the negotiations—until after the deals were done.

A number of studies by commercial consultancies—not all available to MPs—were cited by ministers when promoting the policy, although the studies were roundly criticised when they were later exposed to independent scrutiny. Questions were raised about the use of consultants to develop policy and to defend it, and the extent to which they were substituting the functions of the civil service.

In the event, the contracts, which had been signed with a 30-year term, collapsed within seven years at immense cost to the public purse.

Independent scrutiny?

These events raise the ethical and professional issue of the extent to which executives, civil service officials, and ministers close to projects have licence to be less than fully open, or even allow people to take a misleading impression about the costs to the public purse, or about the state of progress towards completion. 

All large organisations have routine financial audits and controls at all levels to mitigate financial fraud. But where public money is involved, at the risk of creating more bureaucracy, should there be more real­time, independent audits of physical progress and up-to-date cost estimates, transparent to parliament and the public? Any piece of complex technical analysis is at risk of unrecognised or unreasonable assumptions, or straightforward technical errors. It is wise to expose it to independent scrutiny. Large public-sector projects carry political sensitivities, and governments are naturally tempted to over­use “commercial confidentiality” or other reasons to avoid open, public discussion of what is proposed. Circumstances will always change and budgets will always need to be revised, so there needs to be a proper, independently and publicly audited process to manage change. At present, systematic real-time reporting—even if independently done—tends to be kept private when it would be useful to parliament and the public.  

The National Audit Office is independent and has strong powers of investigation, but normally only carries out studies after decisions have been implemented. These are considered on behalf of the House of Commons by its Public Accounts Committee, but any adverse comment that the committee makes usually comes too late and it may not be as feared as it once was. Remarkably, the National Audit Commission’s sister organisation dealing with local government, the Audit Commission, was wound up in 2015.

The UK has the Infrastructure and Projects Authority but, being part of the Treasury and Cabinet Office, it is not fully independent. When they issue “yellow light” or “red light” warnings—as they have done—there does not appear to be much consequence. 

The LU PPP and HS2 were both the subject of several reviews, including those by select committees of the Houses of Commons and Lords and independent bodies. Most of these expressed cautions and some were sceptical of the evidence for continuing. There were many specific suggestions. But select committees have limited powers to discover what agencies do not want them to hear, and often neither government nor parliament are much influenced by their reports. 

Over commitment to projects

A problem for a government is to be able to debate a proposal at a technical and political level in an open way and subject to independent scrutiny, and feel able to modify or abandon a proposal if necessary, without exposing itself to damaging accusations of making a “U-­turn”. The railway timetable change was much more precarious than everybody had been led to believe. Crossrail was not “on time and on budget” as had been claimed. 

Both the LU PPP and HS2 were interesting ideas and worthy of investigation when first proposed. However, over time, evidence and analysis threw doubt on the schemes. Meanwhile, governments had made early political commitments and were determined to push them through in defiance of emerging evidence of increasing cost, longer gestation and construction times, and reducing value for public money. 

It would be a mistake to create unnecessary and burdensome new bureaucracy but it is unlikely that Horizon and the four other cases discussed here are the only ones. If it is too difficult to change the accepted conventions to which people delivering public projects work, are there ways that we can broaden the reach of our existing systems of scrutiny and audit to deal with the problem at proportionate cost?