Why IT Projects Have Cost Overruns (and How to Avoid Them)

Most advice on avoiding cost overruns in IT projects says the same things: plan well, watch out for scope creep and so on. Those points matter, and we cover them here.

We also add what large datasets and our own software development work show about project budgets.

A 2022 study of 5,392 IT projects, led by Oxford megaproject researcher Bent Flyvbjerg, found that a small number of projects account for the biggest cost overruns. The most disruptive components were those with more interdependencies, and an overrun in one of them cascaded to others.

Basics of IT project budgeting

First, a few general project management points that the rest of the article builds on.

Meta-management

Research by Nobel Prize winner Daniel Kahneman and Flyvbjerg’s data on thousands of projects of different types lead to two key takeaways. Anyone running a project should:

1. Understand the planning fallacy: we tend to overestimate benefits and underestimate costs. As Kahneman says, victims of this cognitive bias “make decisions based on delusional optimism rather than on a rational weighing of gains, losses, and probabilities.” The fix is to take an “outside view” and base plans on historical data from projects in the same reference class.
2. Understand the power-law dynamic in IT projects: many projects go over budget by a little, but a few go far over. You need to identify what causes these large overruns and how to avoid them.

   Both points run through the rest of this article. Effective project management needs to go meta: managers must be able to zoom out, see where conventional approaches are not working, work out why and try new techniques.

The triple constraint

It also helps to keep a project management basic in mind: the triple constraint, usually drawn as a triangle.

A project’s quality depends on the balance between cost, time and scope; change one and the others move.

Do you have a software project in mind? Tell us about it!

Common reasons for going over budget

Here are the usual reasons projects go over budget, and what the research says about why.

Downplaying risks

One of the main conclusions of the study above: people underestimate the risks of IT projects. If you are not realistic up front, you put your organisation at risk.

Whether or not the people involved are aware of the risk (and hide it), underestimating it leads to the next problem.

Inaccurate estimates

A proper estimate has to account for contingencies. To win a contract, vendors may rush an estimate, and without proper work up front, that number is little more than a guess.

Even an elaborate plan can be wrong, because the people who make it are subject to the planning fallacy. They take an “inside view” instead of basing estimates on historical data.

Scope creep

Changing the scope during a project has direct, sometimes serious, consequences for cost, schedule and quality, as the triangle shows.

Scope creep happens when a baseline has been agreed, but additions or substantial changes are requested later.

Flyvbjerg often uses home renovations as an everyday example of the same pattern he found in his database of more than 16,000 projects: budgets and schedules are usually exceeded. One of the culprits: changes made after project initiation.

Sticking to the agreed scope keeps the project on track. If a change is essential and you do not want to sacrifice quality, it will cost more money, more time or both.

Lack of communication and collaboration

When things start to go wrong, some team members go silent, out of embarrassment or because they do not know what to do next.

Hiding problems may keep the customer happy for a while, but it costs more later. The sooner an issue is raised, the cheaper it is to fix.

Rigidity

Teams sometimes stick with a method because it worked for another company, even though it does not suit their own culture or mix of cultures.

Following a theory dogmatically when it does not work in practice wastes time and money.

IT project cost management best practices

No method prevents overruns every time, but the practices below, specific to IT projects, reduce the risk considerably.

Get real about risks

The study mentioned at the start of this article points to one lesson: be realistic about risks.

At Go Wombat, we assess the probability and potential impact of each risk.

The study also found that projects with interdependent components carry more risk: when one component goes wrong, it sets off a chain reaction. We explain this to clients whose projects have such components.

We are honest with clients, and with ourselves, about risks, and we use tools such as a risk management matrix.

For more on this, see our article on risk management.

We are very upfront about what it will cost (in time and money) based on our years of experience with these projects.

* Mike Ivanov, CIO at Go Wombat

Do the Discovery Phase

In the Discovery Phase, business analysts assess whether the project is viable: whether it fits your business and the current market.

If the answer to both is yes, planning starts. Two practices make our estimates more accurate:

• Base plans on data: we take the “outside view” and benchmark against the actual costs of previous projects, adjusted for current market conditions. This surfaces work you might not have planned for.
• Break it down: we identify the project’s basic building blocks and reduce risk through modularity, reusing modules we have already built. The solutions architect is closely involved at this stage.

The scope is also defined and documented so stakeholders can refer to it later. Once the scope is fixed, time and cost can be estimated more accurately.

If you want to know more about how Discovery Phase works at Go Wombat, you can read about it here.

Continuously monitor and communicate

On a large project, you avoid losing ground day by day through constant testing and monitoring.

At Go Wombat, we track progress, monitor hours used and flag any likely variance in the hours needed. We send regular status updates, and clients can talk to us at any time, including directly with developers.

It works both ways: engagement from your stakeholders matters just as much.

If bottlenecks do occur, our PMs explain clearly why we need to add more time (and money).

Reporting problems quickly is also what makes the next point possible.

Be agile

This is not only about the Agile methodology. It means being ready to change course quickly when something unexpected knocks the project off track.

Here, one of our PMs, Arnold Skoryk, offers a simple principle: “Locate the cause and adjust.”

Flyvbjerg’s own maxim, from his book How Big Things Get Done (2023), is “Plan slow, act fast”: thorough planning is what makes fast execution possible and leaves fewer surprises to adjust for.

Our PMs are trained in several methodologies and have managed enough real projects to know which one fits where, and when to combine elements of different ones.

Partner with pros

Vet your vendors. Meet them and check whether they have the two things our PMs consider essential for handling risk: experience and vision.

Go Wombat has business analysts, PMs and developers, and 10+ years of experience managing IT projects across different industries. We have handled high-risk projects with many components, such as multiple data providers and APIs (see our article about travel tech).

We are up front about the real risks. If we hit a bottleneck, we tell you quickly and explain why it happened and what needs to be done.

At our size, you get direct communication and a team deep enough to cover your needs.

Flyvbjerg has taken a term from the Greek for what IT budget management needs: phronesis.

Phronesis = deep domain experience + proven track record of success

In other words, practical wisdom.