This is the final post of a 3-part series looking at NCIS, the new customer service and billing system for Seattle Public Utilities and Seattle City Light. This post looks at how the city’s capital budget process is fundamentally broken.
(Here are links to part 1 and part 2)
There is one more lesson from NCIS that needs to be sorted out moving forward: the way the city budgets large IT projects is fundamentally broken. In fact, as Melina Thung, acting Director of SPU, described in an interview last week, the city’s capital budgeting process as a whole is fundamentally flawed, both for IT projects and for buildings.
The central artifact for capital budgeting for the City of Seattle is the Capital Improvement Program (CIP). It defines a six-year schedule for investments in new capital assets. In practical terms, it’s a queue; since there are limited city funds for capital projects, each department needs to get theirs onto the CIP as early as possible to reserve a place in line so that it can be funded when needed — and funds can be planned for and set aside in advance if necessary by the cash-flow wizards in the City Budget Office. Each department has its own CIP; many get rolled up into a big, general one for the city, and some others are kept separate, but in the end they are all approved by the City Council as part of the budget process.
Here’s the problem: when a department adds an item to its CIP, it is required to estimate the project’s budget. But capital projects — whether buildings or IT assets — go through stages of planning to work out what to build and how much it will really cost. None of that work is done at the time the first budget estimate is created to add a project onto the CIP, so that number is almost always invented out of thin air and meaningless. It’s a lovely catch-22: you can’t add a project to the CIP without a budget estimate, and you can’t get the funding to do an accurate estimate unless you’re already on the CIP. And yet, when a project’s budget inevitably increases as the planning work gets done (as happened with both NCIS and the North Precinct police station), the number that is used for comparison is that original, made-up, meaningless number.
This is “gotcha” politics at its worst: the system is rigged so that elected officials can always complain that a capital project is over budget, because they forced the project managers to conjure an initial estimate out of thin air that is guaranteed to be wrong.
Here is the 2011 CIP (approved in November 2010) for Seattle City Light, the first time the NCIS project is budgeted out. A year before customer requirements are first mapped out, and four years before design work is done, they estimated the project would cost $28 million (split 50/50 with SPU).
As a reminder, here are the major planning and design milestones for NCIS:
In the 2012 CIP, the budget estimate had already jumped to $40 million — but it’s still meaningless. In the 2013 CIP, it’s $42.6 million, and in 2014 it’s $44 million. It isn’t until the 2015 CIP (approved in November 2014, still three months before the design phase finishes) that we start to see the budget estimate have any relationship to reality as it jumps up to $62 million, and in the 2016 CIP — the first one with the design phase complete — it’s at almost $90 million. Again: the estimates aren’t wrong because the team is incompetent; they are wrong because the team was forced by the capital budgeting process to make up numbers for years before they had any solid information on which to base real cost projections.
SeattleIT Director Michael Mattmiller noted that Council member Burgess, the chair of the Budget Committee, has asked the City Auditor’s office to review the NCIS budgeting process; the Auditor team does great work and we can hope that they flag this issue with recommendations on how to fix it. But in the meantime, history will continue to repeat itself: capital projects will continue to overrun their initial meaningless budget estimates, and there will be gnashing of teeth and waving of fists in Council chambers.
Council members Johnson and Herbold have proposed a new Council committee to oversee capital projects. However, unless the capital budgeting process is changed to eliminate meaningless budget estimates, it’s unclear whether it will help. The Council has already shown that even when it has previously voted — often unanimously — to approve a project at a higher budget, it will still later complain and blame the executive branch for budget mismanagement.
A better process would institute a set of “gates” for projects, where funding for various stages of planning could be approved before a budget estimate for the full project was calculated. That would take an act of intellectual rigor and political courage: it might look like the city was writing blank checks for planning and design work without knowing the true scope of a project, but it would be admitting that they are already doing exactly that — just lying about it by making up numbers. Barring that, the council could at least change the early budget estimate to a range rather than a fixed amount, to acknowledge the inherent uncertainty.
The NCIS team seems to be on the verge of at least partial redemption, if their upcoming launch goes well. Unfortunately, if the project were begun again next week, it would most likely repeat the same story, since to-date nothing in city government has changed that would lead to a different outcome. Still, there are signs of hope: SeattleIT, once settled in, might raise the bar for large IT projects, and the various eyes on the city’s capital budgeting process might decide to make some changes for the better.
This explains why initial project budget estimates are inaccurate, but not why they are often far too low. Instead of inventing a number out of thin air, why not start with the total cost of the previous billing system replacement, increased for inflation? Or find out what it cost any other large public utilities to replace billing systems in the last decade? Low thin air estimates are driven by wishful thinking as much as ignorance.
A great series of articles on the complexity of these projects and the need to proceed with caution. And thank you for at least the mention of the problems that procurement processes can contribute. Really until the “blueprint” or design phase is complete can you determine the true resource requirements; even then, without a complete understanding of the complexities of the business processes inherant in complex purchased software, there will be surprises that either requrie design change or the need to further modify business rules.
One of the issues is inherant in the fixed price contract model; John Kost, the former Michigan CIO who led Gartner’s Government research group identified the issues with the fixed price contract model, as these bids have to be done far before the scope of the work is truly known, with a competitive pressure to keep the bid low, and provide a time schedule that supports a low price. The public contracting process makes it very hard to revise the contract after the “blueprint” phase, as it is often characterized as the contractor trying to price gouge through a change order; often the reluctance to bring an amedment to the contract before the decision makers further leads to poor practices or a need to rush the project to go live.
While some suggest splitting the project into phases, the need for a second RFP causes delay, loss of momentum (critical on these projects), and the potential to have a new team selected who will need to repeat a significant portion of the “blueprint” to understand the details of the business process that they have to orchestrate. In some jurisdictions, the issue is amplified by a prohibition on the firm that did the plan from bidding on the work (a prohibition driven by a fear that specifications have been tailored for the specific firm).
While the best answer is to limit in a draconian way any customizations of the purchased software, the combination of unique public policies, past practices, and the need to interface with a variety of other business systems can often render this strategy of limited effectiveness.
For those interested, the City of Los Angeles Rate Payer Advocate issued a report on the LADWP Billing System implementation that focused on some of these systemic issues.