Kiwirail’s recent decision to release the full business case for the third main has provided an interesting look into Auckland’s transport alternatives. As Matt outlined in his post, it’s strange that this project hasn’t been funded while the far costlier East-West Link is being fast-tracked:
The section of track between Westfield and Wiri is one of the most heavily used in the entire country. On top of serving both the Southern and Eastern lines, it is also sees more rail freight pass over it than any other stretch of track in NZ. It’s so busy that Kiwirail have declared “the current twin track configuration has reached the maximum operational capacity”“. That lack of capacity is resulting in delays to both passengers and freight. It’s long been known that the best solution to the issue issues between Westfield and Wiri is to build a third track to improve capacity. Yet bizarrely, the project still hasn’t been funded.
To be clear, there seems to be a good case for the third main – its costs are easily outweighed by benefits for freight users and public transport users. And it’s far better than the alternative of shifting more freight to road – that costs more than building the third main yet actually manages to worsen outcomes relative to doing nothing.
However, the business case also provided an interesting demonstration of how a poorly-designed multi-criteria analysis can misinform people about the impact of alternative options. Let’s take a closer look at that – but be warned, this is going to be a quite dry discussion!
This table summarises two different ways of measuring the impact of the project. The top lines show a multi-criteria analysis, or MCA, of the alternative options that the business case considered. As you can see, options are scored on a positive effect / negative effect scale relative to the Do-Minimum option, which is set to zero. Larger positive or negative effects on individual criteria are colour-coded, and then the impacts are added up to get an overall score and ranking.
Below this, the table reports the results of a cost benefit analysis, or CBA, of the options. The benefits of the project for freight users and public transport users have been estimated, and compared against upper bound and lower bound cost estimates. The resulting benefit-cost ratios, or BCRs, show the relative size of benefits versus costs. A BCR above one indicates that the project has more benefits than costs, while a BCR below one is a signal that you should consider doing something else with the money.
Here’s the problem: the MCA and the CBA don’t really line up with each other. The third main option has the highest BCR – in the range of 1.5 to 1.8 – but it’s only the second-ranked option on the MCA. The option to build both a third and a fourth main together has the second-highest BCR – in the range of 0.8 to 1.1 – but it’s the top-ranked option on the MCA. And the option with the third-highest BCR – no rail freight in the commuter peak – is ranked 8 out of 11 on the MCA.
As an economist who carries out these types of evaluation on a regular basis, this is a red flag. If the MCA is so wildly different than the CBA, it suggests that either:
- The CBA has missed some important things that it should have tried harder to measure; or
- The MCA framework hasn’t been designed very well.
In this case, it seems to be the latter. Let’s investigate why.
The first thing to note is that of the nine categories in the MCA:
- Three criteria relate directly to the benefits measured in the CBA – “Freight delays”, “PT Travel Times”, and “Reduced freight on road”
- Two criteria relate to benefits that could be measured as they represent flow-on impacts from the measured benefits – “Safety” (which is enhanced by getting trucks off the road) and “Economy” (which I presume relates to the productivity gains from faster freight)
- One criterion relates directly to the cost of the project – “Affordability”
- All options were scored as having no impact on one criterion, raising the question of why it was even included – “Cultural, social, and environmental effects”
- Two criteria are a bit fluffy but legitimately difficult to quantify – “Feasibility” and “Public / Stakeholders”.
In other words, the majority of the MCA categories are actually linked directly to the quantifiable costs and benefits that should be included in the BCR. This makes it even more puzzling that the MCA results were so wildly different than the CBA results.
We can start to see why this happened when we think about the way that costs and benefits are weighted in the MCA. Taken together, the three measurable categories of benefits included in the MCA get three times as much weight as the “affordability” criterion.
But we know from the CBA that this isn’t realistic! Measured benefits aren’t three times as large as costs for any of the options – in most cases, costs are a bit larger.
Basically, the MCA gets different results from the CBA because it places an arbitrarily high weighting on benefits, and an arbitrarily low weighting on costs. That’s why the fourth main option comes out ahead of the third main option: it has higher benefits, which are amplified by the MCA framework, and also higher costs, which are de-emphasised by the MCA.
Furthermore, a closer look at the “affordability” criterion reveals some serious problems with the way the options have been scored. As you can see, the Do-Minimum is set to zero, as it is the baseline against which other options are considered. The third and fourth main options are also set to zero, in spite of the fact that the cost estimates indicate that they will actually cost more. Meanwhile, the “longer freight trains” and “faster freight trains” options, which cost slightly less than the third main option, is scored as being mildly negative (“-“) for affordability.
It’s difficult to see how spending more can be equally affordable as spending less, but that’s what this analysis implies.
So what does all this mean?
As I said at the start, the third main is a vital project for relieving a bottleneck in Auckland’s transport system. The fourth main also seems like it could be a good idea – if benefits are higher than expected, or costs lower. It’s excellent that Kiwirail is starting to consider when that might be needed. But the results of the MCA don’t prove that we should prefer it over the third main!
More broadly, caution is needed when using MCA to assess projects – or when reading the results of said assessments and trying to figure out what we should do as a result. Used well, an MCA can be vital for assessing options in situations where a proper CBA isn’t practical. But it’s also easy to make mischief with MCAs.
For further reading: the Treasury’s Guide to Social Cost Benefit Analysis has a nice discussion of the strengths and weaknesses of MCA.
Processing...
Clearly an extra track pair should be built.
Firstly, it is cheaper to do one project than it is to do two projects. You save on startup costs.
Secondly, just look at the net present value (NPV). In absolute terms, creating a second track pair has the
highest absolute (note – not relative) benefits.
Thirdly, it is operationally desirable to have separated freight and passenger services each with their own
dedicated tracks. See Perth, Australia.
I’m not sure it’s that clear actually, given that — from what I understand — building a third main wouldn’t require too many structures to be re-built in comparison to a fourth main.
Given the Nats announcement, the focus now has to go on the 4th main. Given how little additional cost there would be to do it at the same time, relative to doing it later.
I dont know you can assume that there is ‘little cost’ for making it 4 tracks. My understanding its a moderate cost for adding one track and a much larger cost for one more again due to the existing rail under bridges.
By all means , where possible, widen the rail track bed, and any streams that have new bridges have the wider abutments ( quite a few existing streams are flood bottlenecks so the stream flow needs to be widened as well).
The same goes for railway stations and the platform connection structures ( underpasses/overpasses etc)
It would be a worthwhile exercise to understand what that cost is now though, lest we end up with a SH16/NW Busway situation again.
It is kind of the Nats to spend the 2 out of 3 tax dollars raised outside of Auckland on us lucky metropolitan city dwellers. The inhabitants of Otago will feel a sense of pride in Auckland’s new bus lanes and the Mill road SH1 bypass. No fear of a political backlash with rural voters unthinking devotion to their MPs.
But that’s just not happening Bob.
What are you basing that comment on??
A belief in user pays. Bring in congestion charges.
The 2 out of 3 was a guess – Aucklander’s do earn more and therefore pay more tax but most export businesses are based in rural areas even if their head offices are here. Certainly the big expenditures are in Auckland whether for accommodation allowances or building motorways that wouldn’t have been needed or been cheaper if properly planned. I may be totally wrong but I bet this is the way non-Aucklanders see it.
You’re correct Bob; that is the way non-Aucklanders see it. Or rather, choose to see it.
Auckland has for decades paid way more in taxes than is spent here. Aucklanders have spent bucketloads on South Island roads.
The current expenditure does not redress that imbalance.
If 2/3 tax dollars raised outside of Auckland were spent in Auckland then Auckland would receive over 80% of all investment. That simply isn’t the case. Auckland gets back approximately what it pays in, and gets nowhere near it’s share based on rates of growth.
Bob, you misunderstood my question. What are you basing your assertion that more tax dollars are spent in Auckland than in other parts of the country?
I assume you would make the same comment about taxpayers from other parts of the country paying for a Manawatu Gorge replacement and the Transmission Gully motorway?
Jezza. That should have been “Some of the 2 out of 3”. The word ‘some’ makes all the difference – and without it of course my sentence is rubbish. As I said I don’t have the figures and even if I did I would wonder what amount of Fonterra is allocated to Auckland -v- non-Auckland income. All a matter of perception. One thing is certain NZ could survive no Auckland but Auckland can’t survive no rest of NZ.
The government has decided to go for votes and who can blame them. Rural electorates are usually solid National but offering baubles to Auckland voters may win them the votes needed to win more seats in Auckland.
Bob, who is going to pay for all the damage being done by Fonterra’s palm-kernel factory farming stupidity? Who is going to pay for the health problems associated with car dependency? Who is going to pay for the tourist industry’s damage to our natural environment?
The imter-generational problems here are at least as big as any perceived regional vs city problems. If NZ can’t manage to see past this sort of division then we are doomed. Let’s try to solve problems as a country, not as a set of bickerimg regions.
“One thing is certain NZ could survive no Auckland”
Sure, with a significantly reduced quality of life. How would we maintain Westland’s roads without the huge subsidy from Auckland’s tax payers? Who would buy all of Malborough’s wine? Who would run the logistics for Fonterra? Where would the big chunk of tourist dollars in New Zealand that no longer exist come from?
Bob – you have still not presented any evidence that more money per capita is spent in Auckland compared to the rest of the country. The government has (re)announced they are spending $260m per year in Auckland, I imagine they will spend considerably more per year across the rest of the country. The transport budget is more than $1 billion a year if my memory is correct.
Also I’m not sure you have a grasp of MMP, every vote counts across the country. Someone in New Plymouth who is deciding between National and the opposition has the same influence as someone in Auckland.
Bob. Here’s the CRL funding for example: 50% AKL ratepayers, 50% nation’s taxpayers, of which just over 1/3 are Aucklanders.
Therefore Aucklanders pay 50% + (35% of 50%) ≠ 50% + c17%
Auckland’s share = around 67%
No other part of the country pays any local contribution to strategic transport investments, except I think WGTN did pay 10% towards rail upgrades, all state highways are 100% central funding. Which of course means over 1/3 of everyone else’s kit is funded by Aucklanders.
“all state highways are 100% central funding”
That’s not true. Waikato DC and Hamilton DC both contributed to the Te Rapa Section of the Waikato Expressway and Taupo DC contributed to the East Taupo Arterial.
How much?
ETA: 25%
http://www.taupodc.govt.nz/our-council/policies-plans-and-bylaws/district-plans/changes-to-the-district-plan/plan-changes-28-33/plan-change-hearing-documents/graphics/Documents/Graphic-39-East-Taupo-Arterial-newspaper-special.pdf
Te Rapa Section 8% HCC, 2% WDC
https://www.nzta.govt.nz/projects/waikato-expressway/te-rapa/
Excluding tourists what percentage of users of the CRL will be non-Aucklanders? My guess 95% . If I am right then outsiders are subsidising it (by 28%). We ought to do it ourselves with a toll on the bridge and Waterview.
I completely agree, Wellington Region should be forced to pay for the $3b motorway north themselves too. And Canterbury for their $3b motorways north and south. And the Waikato for their $3b motorway. And the Bay of Plenty for the TEL.
Or we could recognize that these are nationally important and fund them nationally.
Auckland would benefit hugely from local funding of roads and PT. If we took that approach the residents of the Waikato District (Pokeno, Huntly, Ngaruawahia) would have to fund most of the Waikato Expressway despite the majority of users being from Auckland or Hamilton.
If the Wellington motorway is only used by Wellington residents then they should pay for it. Rather obvious. But if it is a strategic advantage to every Kiwi then we all should contribute. How to split the tab is worth discussing.
Given modern technology not too difficult to implement a usage charge (as per the tunnel before Puhoi). I’m not against the CRL and I’m willing to pay for it as an Aucklander (even though I rarely cross the bridge from North Shore) because it is a benefit to the whole of Auckland – reducing road usage etc. However outsiders will not use our CRL and road congestion in Auckland is none of their concern. Why should they pay for it?
I completely agree that we should have far more user pays infrastructure (the Orewa to Puhoi tolls don’t even cover interest on the loan we needed to build it, btw). However, it is unreasonable to exclusively target the CRL for this. Every region has several comparable projects. We should, over time transition much more to user pays for state highways and major regional transport infrastructure.
Sailor person: I suppose the CRL is only about 50 years late. Today we are promised a busway – should have been built when the motor way was built of course but the Nats announced finance today – I’ve not read their announcement only heard about it on the news but it did sound as if the gov has suddenly found a lost piggy bank. A busway west only helps Aucklanders but to the airport it helps others too. Funding should be appropriate.
Joining in this dry discussion – I prefer a slightly different approach to MCA I learned a few lifetimes ago.
Each criteria has an
Essential
Preferable
Desirable
category. For example, for Freight Delays it might say:
Essential: 2% improvement in time from X to Y
Preferable: 2.5% improvement in time from X to Y
Desirable: 3% improvement in time from X to Y
Then they are scored. Because “Essential” is the most important, it might get 50 points, preferable might get 20, desirable 10
If Option A reduces time by 3% it gets 80 points (50+20+10), if Option B gets 2% reduction it gets 50 points, if Option C gets 1.5% it’s automatically ignored because it doesn’t mean the minimum essential score
This requires explicit criteria be included, rather than waffly fuzzy huggy hippy bourgeois crap. At the end, you get a score for a project, and you divide it by cost. Easy. Of course like all things this can be gamed by including a whole heap of additional criteria that simply out-weigh the ones that really matter, which is why you need to clarify requirements before embarking on this sort of analysis. First set your goal!
Also, affordability should NEVER be in these criteria because *cost is a denominator*. It is never a numerator because cost is never a benefit.
So why bother doing an assessment if it just relies on what someone chooses is essential or preferable? You say you don’t want waffle or fuzzy, so instead just arbitrarily decide what you want up front, which is worse than being flexible and relative. Why pretend you are dressing it up as impartial science or something.
Oh and cost can often be a benefit, you can absolutely have operating cost and capital cost savings versus doing nothing, do minimum, or doing an existing programme. A lot of public transport projects have net cost savings vs. the status quo, as they save a lot in operating expenditure from small investments in capital expense. Sorry if that’s too waffly for your tastes, but unfortunately reality is never so black and white.
When performing BA on a proposed IT solution your approach has merit (though always with a lot of background/justification for the scoring), however I’m unable to clearly see the merit in this context. Perhaps I’m missing something?
Just checking, to help my understanding did you mean:
Basically, the MCA gets different results from the CBA because it places an arbitrarily high weighting on benefits, and an arbitrarily **high** weighting on costs. That’s why the fourth main option comes out ahead of the third main option: it has higher benefits, which are amplified by the MCA framework, and also higher costs, which are de-emphasised by the MCA.
or did you mean “low” in that asterisked spot?
Yeah, I’ve edited to fix that.
How do the ART3 model results feed into all this? Am I right to assume those predictions are used as a basis for this analysis too?
I’m serious here. For road projects the modelling process is so very flawed that the regional ‘person trips’ are considered the same whether the project is included or not. The model cannot produce ‘newly created trips’ despite the overwhelming evidence otherwise, and this makes roads look erroneously good in the BCA.
Is this true for rail too? Is the amount of freight and number of passenger trips kept the same regardless of whether there are 2, 3 or 4 tracks?
Of course I doubt this is what they would do, but if rail and road are treated differently, why?
probably not. ART is mainly about passenger movements. Commercial road vehicles are included, of which freigh is a component, but not rail freight.
I suspect Kiwirail have their own rail freight demand model that maybe draws on (at a high level) the national freight demand work undertaken by the MoT and NZTA.
Thanks, Stu. It would be great to find out more.
With the Waterview Connection, the modellers didn’t like the results for heavy vehicle movements so they adopted 45 pc of that figure. They have since ‘fixed’ the error in the model but the public don’t know what else the error affected.
Freight is of course a big part of the EWL modelling, and passenger trips should be a big part of the third and fourth main analysis, with consequences for travel demand on motorways.
The modelling needs to be multi modal and follow the huge steps forward in modelling that are happening internationally.
Perhapd KR is further ahead on this than NZTA? The economic analysis is meaningless without a good transport model to base it on.
Heidi? ‘What will the effect of the 4th main be on the number of heavy vehicles servicing the Ports of Auckland? Will it reduce the number of heavy vehicles in that area?
I’m off topic but could I ask if there is a way to model and gain CBA for increased parking costs in the CBD if that were say to be increased to the point at which 15% of spaces available at any time?
I ask because my feeling is it would reduce the volume of peak hour traffic and increase the Public Transport useage.
Hi Ted, I would hope getting some trucks out of the CBD would be one of the benefits of the third main that most people can agree on.
There are certainly integrated land use models that cover all modes and from a quick search it looks like places that really value their inner city land, like Germany, try to model the dynamic effects on inner city land use well. I don’t know if parking charge models are ever included or if these places have long since sorted the too-cheap parking issue.
What I noticed from a search I’ve just done is how difficult the modelling must be for landlocked cities, particularly those where surrounding areas are in another state or country or transport plan. Modelling well in Auckland should be so much easier.
I’d love to hear more from someone who works in the area.
National have just announced an election sweetner – including confirmation of 3rd main and electrification to Pukekohe. Which is good. But while the 3rd main is costed at between $65-80m here, Bridges has said he is allowing $100m in the package – reckon that includes some of the work for the 4th main? Or is he already picking a price inflated by 20% or more?
“He said the projects would involving substantial new work across the city, designed to be completed over the next decade.
The new southern highway, along Mill Rd, was costed at $955m, with the Northwestern Busway at $835m. The package also includes $616m for AMETI, $130m for electric rail to Pukekohe and $100m for the new rail line.
The total package was worth $2.636b.”
https://www.stuff.co.nz/auckland/95437942/national-says-it-will-spend-billions-to-combat-aucklands-traffic-and-transport-woes
Guy this is simply Bridges taking everything in the first decade of ATAP, i.e. the agreed forward programme between the Gov and the Council for the next ten years and putting a big blue ribbon around it and calling it a gift to Auckland.
Don’t get me wrong, it’s great that after 8 years of governing they finally sat down with the country’s biggest city and fastest growing economic area and have now agreed to allow our taxes to be spent in a coordinated way with our rates on some desperately needed transport projects (mostly), but this isn’t news. Not new news.
But great to get funding confirmed, would be got to get timing confirmed too. As the timing has very real impacts on other decision such as battery trains etc…
Yeah – I missed that until you pointed it out. Perhaps JAG or Jacinda should be pointing that out in public, too.
Well if BEMU aren’t needed now then that saves AT what was it $100m? That should certainly help keep the AC budget under the threshold and not require AT to rob Peter to pay Paul. That is of course assuming that the plan is to do this in the next year or two of course.
It was about $0m as they still have to pay for new EMUs regardless.
No it wasn’t at all Sailor Boy. The battery packs were potentially about $100m more additional cost than just EMU. Sure the costs of electrification are expensive but that is the long term solution needed anyway and is permanent.
Sorry I missed the ‘4’ I meant to say $40m, but that was also wrong. It’s somewhere between $40m and $70m:
https://www.greaterauckland.org.nz/2017/07/24/battery-trains-approved-pukekohe/
i imagine building the third main to be like making a trip to the supermarket for 1l of milk (enough to last a day) while building the 4th main would be like making the same trip to the supermarket but getting 3 or 4l of milk (enough milk for the rest of the week).
Luke, to extend your analogy, you might cycle to the supermarket for your 1 litre of milk but find that 4 litres is too much to carry on the bike so for your weekly shop you have to drive. Same trip, different cost structure.
Id probably look at a compromise where I took 2 litres home on the bike rather than 1 then and the second trip has been delayed.
I say bring back glass bottles and home delivery.
If you use large pannier bags then with the low centre of gravity you will hardly notice whether you have 2 litres or 8 litres on board your bike.
There are certainly sections of the 4th that look like they would be both cheap and extremely useful. For example Westfield junction through Otahuhu station looks like it has a clear RoW already and would entirely separate Freighters from Metros at this pinch point… the 3rd is already there on the west side for Metroport, and the 4th on the east side would allow separation of PoAL freight traffic too…
So basically get the 3rd main and build parts of the 4th as passing loops (plus improve certain junctions). That would explain why it’s coming in at $100m rather than $60odd million.
There was 50m in ATAP for Westfield junction improvements which is basically most of that so they possibly just added them together.
If your MCA doesn’t align with your BCA then it may well be that your BCA has failed to account for all benefits and costs. I’d definitely say that is the case here, where it looks like they’ve only looked at Westfield to Wiri instead of Britomart to Pukekohe.
Also your assessment that “Taken together, the three measurable categories of benefits included in the MCA get three times as much weight as the “affordability” criterion.” isn’t quite right. Feasibility is also a cost. Anything is feasible, it’s just a question of how much money you have to throw at it.
I agree that there are major issues with the MCA here, but there are other things we need to talk about within that.
Agree that is the real issue hard to nail down true benefits of fourth main as main benefits outside scope of report.
Yes if the 4th main can give us express & interregional trains from further south, the benefits could be huge.
‘Affordability’ is a very odd and unsatisfactory criterion. The report nowhere explains what this is supposed to mean and how it is to be judged.
Is it supposed to relate to absolute cost *relative to some budget constraint*? Is the idea that if options A and B have exactly the same BCR, but B is a bigger project (all the cost *and benefit* figures are bigger by the same proportion), that gives B a worse score on ‘affordability’, because it’s more likely to hit a budget constraint? That’s a fair thing to consider, but it would be nice if the report explained itself properly.
I question the utility of a criterion like this in MCA. Let the consultants calculate the costs and benefits, and let the government decide what it wants the budget constraint to be having regard to its other priorities. For the consultants to mark down an option on ‘affordability’ is prejudging the outcome.
The safety measure probably isn’t used because it will be for the project itself. I.e., a road upgrade evaluates safety improvements delivered by upgrading the road. That measure here would only have looked at how much safer the railway becomes as a result of the upgrade, and I suspect they deemed it to not be relevant as it’s already a no-go zone for the public, and safety is already about as good as it can be.
Great photo by the way. It looks familiar!