The private and public sectors both suffer all too often from expensive and embarrassing failures of project and change management. A couple of private sector examples are mentioned here. Although drawing heavily on government and government-related publications, this part of this website offers advice which should be relevant to senior managers in all sectors of the economy.
This area of the website contains the following key pages:
- This introduction and key advice
- Project Initiation and Planning
- Managing Change
- Appraisal Optimism
- A Project Management Checklist
- NAO - Lessons Learned from Major Projects
- NAO Survival Guide to Challenging Cost Estimates
- De-risking IT Projects - Advice for Non-Experts
What Can Go Wrong
The British government spends an unfathomable £670 billion every year – and often does so in ways that are wasteful and misguided – So begins the cover blurb for Conundrum – Why Every Government Gets Things Wrong by Richard Bacon MP and Christopher Hope. The book contains a depressing and all too familiar catalogue of badly designed and implemented major programs, but its central thesis is that delivery is too often the poor relation to policy for both Ministers and civil servants. The following advice does not aim to remedy all the faults identified by Messrs Bacon and Hope, but – if you were to follow it – it would certainly improve matters.
To begin at the beginning - the words ‘program’ and ‘project’ are, in most cases, interchangeable. The key thing is that they conceptually follow on from a policy decision – a decision that will hopefully have taken account of the practical, financial and other factors that will constrain the deliver of that policy. In theory, a program is large scale, and often consists of a number of projects. But a project can be pretty big too, and can certainly appear very challenging to those in charge of it. Programs and projects should therefore be managed in pretty much the same way, using the approach outlined below.
But it is first worth noting that the initiation of a major government project can be a very sudden affair. Presidents and Prime Ministers feel under great pressure to announce dramatic goals. Some of these work out OK, such as Kennedy’s 1961 belief “that this nation should commit itself to achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to the Earth”. Neil Armstrong stepped onto the Sea of Tranquility in 1969. Some don’t turn out quite so well. I’ll leave you to think of your own UK examples.
And many lesser mortals can fall into the trap of what the World Bank tactfully calls appraisal optimism – the presentation of best case scenarios including under-estimated costs and timescales, and over-estimated benefits, in order to get projects started.
Mainstream officials are more likely to be heavily involved in the policy development and project definition phase of major projects and then acting as the principal customer of a specialist project/program manager. Indeed, significant projects should always be led by someone with proper project management training and experience, and the civil service has recruited a good number of these in recent years. So the first thing to get right is the ownership of the project.
The 2019 report on the TSB banking systems failure makes it clear that neither the Board nor the delivery team understood the challenges, and that no-one had been given clear responsibility for it..
"5.15 ... TSB has told us that the entire BEC was responsible for the Programme ... It is clear [to is] that the TSB CEO, the TSB CIO and the TSB COO were responsible for leading this Programme."
Why Projects Go Wrong
The National Audit Office has listed the following perennial problems:
- Civil servants may be reluctant to highlight unrealistic timescales or the need for further pilots and planning, as they want to be seen as ‘can do’.
- This can generate pressure to hurry the implementation of reforms and projects.
- However, sustainable changes to public services needs a strategic perspective, careful planning and sound implementation.
- If a department can only move at a slow pace, it can be detrimental to value for money to pretend otherwise.
- A department may also only be able to take on so much change at once.
- There is no deliberate resistance to Ministers’ demands. Instead, there is unacknowledged reality.
And here is the NAO’s summary of what happens when officials ignore inconvenient facts – as they do:
- There is pressure to present the unlikely upside as the most likely result.
- Major projects under-deliver on benefits and overshoot on time and cost.
- There is optimism-bias.
- Changing requirements lead to delays and increases in costs, for example in major defence projects.
There is more information about such appraisal optimism here.
It follows, therefore, that you should do your best to persuade an enthusiastic manager or minister that more haste equals less speed, even though they may feel that you are dragging your feet and/or seeing unnecessary obstacles. They may however recognise the thought that there is a natural rhythm to a good project – nice and slow and cautious at the beginning, so that rapid pre-planned progress can be made at the end. It is a trite but accurate observation that a new building will soon be finished after its foundations have been laid. Ministers will still need to make those big bold announcements, if only to forestall the announcement of a similar ambition by a rival. But such announcements should explicitly allow you time to research and plan the project in some detail.
Pre-announced end-dates, in the case of more complex programs, can be deadly. Crossrail (now the Elizabeth Line) was a classic example. Its construction began to run late mainly because of complex railway issues, as distinct from civil engineering issues with which its senior team were more familiar. According to Crossrail's later CEO, Mark Wild, the whole project should have been re-sequenced in 2016 but "it would have taken a brave person to do this", even if the need had been recognised. In the event, Mr Wild believes that the delay in re-planning the project probably added a year to the timetable and cost £1billion.
Bruce Webster coined the phrase 'thermocline of truth', nicely explained in this commentary on his post:
'How is it that projects seem to be steadily on track for successful delivery until just before the target date for implementation – at which point they suddenly aren’t? It seems implausible that the phenomenon should be helpfully explained by reference to temperature gradients in oceans, but that is exactly what this post does. For all sorts of reasons – some covered in the post – reporting systems give false assurance to senior decision makers for far too long, until the tipping point is reached at which the actual state of readiness can no longer be concealed, when suddenly the project dashboard goes from benign to catastrophic in an instant. Crossrail provided a particularly spectacular and very public example when it failed to open in 2018 having been presented as being on track to do so only a few months earlier – despite what we now know to be a need for more than three years further work before the line could open.
But this is not a phenomenon limited to big infrastructure projects. I can readily think of examples at more or less every scale, and I suspect anybody with any familiarity with project delivery will have their own. From that experience, I see another critical factor encouraging the development of thermoclines of truth: the setting of delivery deadlines. The further ahead the deadline is set (and the more arbitrary it necessarily is as a result) and the more visible that deadline is, the more certain it is not just that the deadline will not be met, but that the fact that it will not be met will only be recognised at a very late stage. There are some pretty obvious reasons why this adds to the risks Webster identifies in his post – the stronger the commitment, the more visible the failure, and the greater the reluctance to face up to the problem, still more so to be the messenger of bad news.
Delivery happens when the work is complete. Completion of the work does not happen because delivery is due. '
Be aware, too, of the Hawthorne Effect. It makes a lot of sense to test or trial major changes in a small part of an organisation or system. But those involved in the trial will almost certainly perform somewhat better, and/or accept change more readily, than those involved in the later roll out of the innovation but who are not in the spotlight, so to speak. The improved performance is unconscious or subconscious, but you do need to be aware that performance during the trial may not be replicated in any later roll out.
There is a related problem in that test runs and trials are often well funded. There is no point in declaring them to be successful if the same resources - including management resources - are not devoted to the subsequent national roll out.
There is another trap, in that some projects look to be obviously sensible, when a bit of research would have shown the opposite. The classic example is the 1970 Scared Straight! projects in the States, where juvenile delinquents were exposed to inmates in local prisons with a view to persuading the kids to turn away from a life of crime. Nearly 40 years on, these projects are still being started, although it is clear that they increase criminality, not reduce it. The original survey and statistical research was badly flawed.
Click here to access a sensible checklist to help you plan a large project. It includes nine key questions that must be answered before there is full commitment to the desired objective.
Change Management ...
... is a subset of project management. These projects involve changing the culture and structure of organisations, and they are therefore especially challenging. Some advice is here.
And anyone involved in a sizeable project should read the NAO's 2020 report Lessons learned from Major Programmes.