How transport projects are evaluated has always been of interest to me. I believe that although the standard cost benefit analysis approach that lies behind the NZTA economic evaluation manual has its flaws, the resulting BCR is still an important factor in determining whether a project, or a particular project option, should proceed. I don’t really buy the argument that a project with an unfavorable BCR should be trumped by “strategic” reasons to enable it to proceed. If the strategic reason is any good it will probably be reflected in the BCR, particularly if wider economic benefits (WEBs) are taken into account. I put the Puhoi to Warkworth business case in this category (BCR 0.92) , along with the eye-wateringly expensive Additional Waitemata Harbour Crossing for cars and trucks (BCR 0.4).
Recently I took a look at a number of documents on the East-West Connections project – formerly called the East-West Link, released by the NZTA . At this stage they’ve completed an “Indicative Business Case” (IBC) – essentially, an initial investigation of the options for improving connectivity in the area. They’ve published the IBC alongside a number of technical appendices.
This is a welcome step as this is the first time that the wider public is getting a decent look at the project, including all of the options on the table. NZTA’s decision to release the full documentation, without redacting large sections of the analysis, is really good for transparency.
So let’s take a high-level look at some of the specific trade-offs between costs and benefits of the different options. To jump right to it, NZTA’s conclusion is that Option F should be progressed to a Detailed Business Case. Here’s a picture of Option F, which involves a new highway along the Onehunga foreshore:
For context, Options C, D and E also involved building new roads (part of the way) along the foreshore, while Options A and B entailed upgrades to existing roads, including freight lanes. A summary assessment of this “short list” is available in Appendix O of the business case.
NZTA conducted a cost-benefit analysis (CBA) of these six options. CBA for transport projects typically compares:
- The financial costs to build and operate the project, and
- The monetised economic benefits of the project, including user benefits such as travel time savings, vehicle operating cost savings, and reliability improvements and other benefits such as vehicle emissions reductions (or increases) and effects on economic productivity (“agglomeration”).
While CBA does have some weaknesses, largely due to shortcomings in the modelling tools available to us, it’s a conceptually robust way to assess project options. Especially in the case of road projects, NZTA’s approach should capture the majority of the economic benefits arising from projects, including productivity improvements for freight users.
This is the summary table:
You can see the net present value of the total benefits exceed the total costs for each option – i.e. the total benefit-cost ratio (BCR) for each option is above 1.
However, there are two problems.
The first is that the NZTA has made somewhat arbitrary assumptions about agglomeration benefits, which in theory reflect the productivity gains arising from improving connectivity between businesses. Rather than formally modelling it using the procedure specified in Appendix A10 of NZTA’s Economic Evaluation Manual, they’ve simply assumed that agglomeration impacts will add 25% on top of transport user benefits for each option.
As Stu previously highlighted in the case of the Mill Road highway, which included a similar “fudge factor” for agglomeration benefits, there is no real reason to do this (other than the rather circular argument that the same thing is being done for other projects).
In the case of East-West Connections, there is a stronger argument to be made for agglomeration benefits, as this project will serve a busy commercial/industrial area. However, it’s still necessary to do the analysis to establish their existence and magnitude! I (perhaps cynically) suspect that the 25% figure has simply been used to make the BCRs all appear higher than they otherwise would be. For public relations purposes it is preferable to have a higher BCR than a low one, even though the purpose of the exercise is an option evaluation rather than an assessment of the absolute economic worth of the project.
The second problem with this table is that it is not consistent with NZTA’s own requirements. Section 2.8 of the EEM sets out requirements for calculating and reporting BCRs. That section requires an incremental analysis of costs and benefits:
In other words, if you are choosing between two options, one of which is considerably more expensive than the other, it’s not enough to say that the more costly option has a BCR above 1. It’s actually necessary to show that the added (incremental) benefits of the costly option exceed the added (incremental) costs.
This is an important step in cost-benefit analysis as it shows you whether spending that extra bit of money for a more expensive solution is justified. Failing to do an incremental CBA is basically an invitation for gold-plating and overspending – i.e. find a worthwhile project, and then jack up the costs as high as possible.
So let’s take a look at an incremental BCR analysis of the East-West options. For simplicity I’ve focused only on Options A, B, and F – the two cheapest options, and NZTA’s preferred option. (Options C, D, and E are fairly similar to F in terms of total costs and total benefits – including them wouldn’t get a different result.)
Here’s a picture of Option A, which is an upgrade of SH20 and the existing Nielson St route to SH1:
And here’s Option B, which is pretty similar but also adds a south-facing ramp to SH1:
I’ve ranked the options from least to most expensive:
- Option A has total costs of $200 million and total benefits of $850 million. Consequently, it has an incremental BCR (relative to spending nothing) of 4.3. In other words, Option A seems like a good project.
- Option B has total costs of $500m and total benefits of $1650m. This means that it has incremental costs of $300m (i.e. $500m-$200m) and incremental benefits of $800m (i.e. $1650m-$850m). Its incremental BCR, compared to Option A, is therefore 2.7. This suggests that it’s well worth spending the extra money for Option B.
- Option F has total costs of $800m and total benefits of $1550m. Relative to Option B, its incremental costs are $300m and its incremental benefits are -$100m. Its incremental BCR is therefore -0.3.
In other words, if the NZTA were to follow their own economic evaluation manual it shows that Option F is not great value for money. It costs a lot more while actually delivering fewer economic benefits than Option B. Negative BCRs are generally not a positive sign that a project is a good idea.
This isn’t to say that we shouldn’t build Option F, or that we should build Option B. There may be some significant positive or negative effects that aren’t captured in this analysis and that may tip things in a different direction. For example, existing traffic modelling tools may not capture travel time reliability benefits very well. Similarly, we haven’t taken a look at environmental costs – on the one hand, Option F paves over the remainder of the Onehunga foreshore, which is negative; on the other, it potentially moves trucks away from the town centre and residential areas, which might be a good thing.
Admittedly I’m not a professional economist, but to me the incremental BCR analysis does highlight several questions that need to be answered:
- Given the fact that Option F costs more than Option B while delivering fewer quantified economic benefits, is there evidence that other unquantified benefits, such as travel time reliability, are sufficiently large to justify the added costs?
- Given that the project is primarily intended to improve convenience for freight users, has the government asked freight companies and shippers in the area if they would be willing to invest their own money to pay for Option F?
- Given the results of the Basin Reserve Flyover hearings, in which a Board of Inquiry found that the incremental economic benefits of the Flyover weren’t sufficient to outweigh the added environmental/amenity costs, is there a risk that approval for Option F won’t be forthcoming?
And finally, given the results of an incremental BCR analysis, isn’t there a case to also progress Option B for a more detailed assessment in the next stage of the work, given that it maximises economic benefits at a lower cost?
Option F would stop the buried toxic chemical mess that is Pikes Point from becoming a serious environmental problem – form a wall between PP and the sea.
Why build a motorway, why not just a wall?
Good work Cam. It seems highway promoters are routinely in the habit of arbitrarily cranking up inputs in order to get a pleasing looking result, and it takes no small amount of work to keep track of it. Here is a transport analyst on the fun and games that is the massive, and massively destructive, Westconnex project in Sydney:
Yes monetary travel time savings are overstated because commuters travel on their own time, not their employers and they don’t value it at the rate that they are paid. There was a survey not so long ago that found any travel time savings would mostly be spent not on extra work, but on increased family / leisure time. I also recall that tolling of the Western Ring route was looked at (I think $2 for a 10 or 15 minute time saving) but the modelling suggested 50% of the traffic would still take the free longer route, hence tolling never became an option.
This is not to say that increased family and leisure time don’t have value… just that it won’t be reflected in economic metrics like GDP.
The SMH article linked to above says this about value of increased leisure time:
Read more: http://www.smh.com.au/comment/the-way-we-justify-investments-in-road-projects-like-westconnex-doesnt-add-up-20151129-glaq7g?skin=smart-phone#ixzz3tDBF11TL
Follow us: @smh on Twitter | sydneymorningherald on Facebook
Anyway this still ignores that great big elephant in the room of driving travel time savings called induced demand. How long will any travel time savings actually last on these projects? If the road is of any utility [and why build it if it is not?] then it will generate new trips that soon clog it up slowing traffic and wiping out saved time. It is likely that these notional savings are entirely illusory too.
Just to issue a qualified defense of the evaluation methodology… common practice is to value non-work trips at a discount on the average hourly wage. I think NZTA’s approach values commute time at something like 40% of the average hourly wage.
That being said, observed willingness to pay for incremental time savings can be quite small.
But is commuting considered to be a non-work trip? Which it should be, I think,as Cam says. Work trips should only be people travelling as part of their job, during the working day.
yes, commuting is distinct from travel for work purposes. The latter is on work time; the former is on personal time – and hence has a lower value ascribed to it.
I can’t understand why they haven’t included an exit from the container terminal across Hugo Johnson Drive to Sylvia Park Rd using the unused freezing works site.
What a criminal waste of money Option F is.
Looking at the incremental costs and benefits is just common sense. I had already done as much in my head when looking at the table before reading the rest of your post. So – how did they address this in their report (assuming they did)? Its not even like option F has higher benefits than B but that incremental costs exceed incremental benefits – the absolute net benefits are highest for option B.
Surely they didnt rank one as being more strategic than the other, they are both doing the same thing??
The route was decided back at the end of 2013, everything since then has been about trying to justify it. Same thing is happening with AWHC.
OK so just bullshittery you reckon. If I was an economist I would be bloody annoyed at the ongoing debasement of my profession.
Have now glanced through the report. I see it is a total jack up. They dont like Option B because it results in more traffic on Neilson St. Well, given Neilson St is an industrial area, what is the problem with this that isnt captured in the CBA? They also talk about travel time issues with local access – again why can this not be captured in the CBA? There is also something (a line item in the transort criteria matrix) about the quality of the Onehunga to Sylvia Park connection. WTF? Why is that not captured in the CBA? That almost seems like people are going to just plain like driving on the new road better than an upgraded Neilson St. They are also worried about encroaching into Hamlins Hill but I cant see the details around this. It is a fairly manky park and I am surprised they could spend some of the $400m of additional benefits/ reduced cost on mitigating that.
It isnt really a debasement of economics as they dont appear to have twisted the CBA, just ignored it. What a disgraceful report.
I’m surprised to hear you’re not an economist. You always sound *exactly* like an economist!
Basically, you’re pinpointing the contradictions and overlaps between a “strategic fit” assessment and a cost-benefit analysis. In principle, strategic fit could be used to ensure that things that are difficult to quantify don’t get excluded from analysis. For example, induced land use change is hard to model. So you could include a strategic fit criterion asking whether a project is likely to enable a change to higher-value land use, based on outcomes previously observed after similar projects.
In practice, a lot of the “strategic” criteria are already reflected in transport CBA, or could be included with minimal additional effort. Reliability improvements, for example, can be modelled or estimated and subsequently quantified as a line in CBA. It’s not necessary to undertake a qualitative assessment of whether a project *feels* like it should improve journey time reliability.
They also talk about the limited “endurance of benefits”. Well – that will be included in the CBA.
‘endurance of the benefits’? Is this an acknowledgement of induced demand?
And some options induce less? Wouldn’t that just mean they’re not as good?
Yes I dont know if this actually means they believe the CBA is in error? It is very hard to tell. And if so why did they not caveat the CBA results.
Wouldn’t that be a good thing? The stated aim of this project is to allow better freight movement. Clogging up the road with SOVs is surely something that won’t help this outcome.
The deal was well and truly done in 2013.
Iwi were told a load of half truths about harbour water quality but not told the proposed motorway will make environmental matters worse. There are no recent reports that show the old Pikes Point tip is leaking, or will leak.
What I do like is the Port of Tauranga removing the truck counter from their web site. It made a mockery of what NZTA and the corporate spin doctors would have us believe about heavy traffic flows in the area.
The sad part for me is the AA, an organisation that I have paid my fees to for years, and one I believed had the interests of New Zealanders at heart, is all for this $300,000,000 a kilometre strip of corporate welfare.
“This isn’t to say that we shouldn’t build Option F, or that we should build Option B. There may be some significant positive or negative effects that aren’t captured in this analysis and that may tip things in a different direction. For example, existing traffic modelling tools may not capture travel time reliability benefits very well. Similarly, we haven’t taken a look at environmental costs – on the one hand, Option F paves over the remainder of the Onehunga foreshore, which is negative; on the other, it potentially moves trucks away from the town centre and residential areas, which might be a good thing.”
I guess one other consideration is that Option F is effectively creating new land in Auckland (on otherwise wasted area of polluted harbour inlet), while Option B is taking it away (from business land to roads). Not $1B worth of land mind you but still the land has got to be worth quite a lot and Auckland is short on land.
This land is so highly valued, that the best purpose it can be put to is parking (second hand Jap) cars awaiting delivery to car yards.
Fact is much of the land there is low value “scrap yard” type land, low-value (adding) enterprises.
And its not like that part of Auckland is short of that sort of land. In fact its overrun with it.
And the former Co-generation plant that Contact owned which is adjacent this land, will also be reverting back to industrial land, adding even more to the stockpile.
Option F doesn’t create any new land or add access to land which needs it. This Motorway is a “limited access arterial” which means its 1 grade below a full on motorway, no real on or off-ramps to speak of.
A very few landowners may benefit from having a access to/from this road, the rest of us – not so much.
It also puts a large barrier between the Onehunga harbour foreshore and the rest of Onehunga and public at large. Way worse than SH20 does now for the western side of Onehunga.
Who wants that? for the longer northern portion of Auckland second harbour?
Hi Greg, appreciate the humour 🙂
I’m not saying go with option F, just pointing out that land is also a consideration. My understanding is that they will be reclaiming land with this option (including for a bike-path).
You are correct that this land is currently low value (mostly due to it’s industrial nature and congestion) but as we all know adjoining Onehunga was until recently also low-value but is now close to being prime-real estate! Likewise Wynyard was also a wasteland but is now prime land.
I’m not saying that this industrial area will be prime land but that it’s value will almost certainly rise considerably due to it’s fairly central location, good connections (with any of the 3 options above) to 2 motorways along with the inland port and rail connections. Also with the eventual airport rail line and possible LR connections to Onehunga from Dominion Rd etc this will only drive up the value of the area.
What Option F does do is as you point out provide motorway lite roading but does also free-up existing roading (not that that is necessarily a good thing – unless the existing roading becomes less of a non-pedestrian place).
That part of the harbour has never really been good access for Onehunga and isn’t very pleasant at all for the most part. I certainly wouldn’t be swimming in it even if they went and made beaches along there as that area is polluted and heavily contaminated.
Someone suggested on here recently that they should just dam up the inlet from the bridge and fill it in for land. Not a bad idea in many aspects. 🙂
strongly agree fill in the harbour right up to the bridge
Well that will presumably already be included in the CBA. Costs of land aquisition will be included in the other options. In fact the issue you have brought up may actually properly count against Option F, as the assumption probably is that the reclaimed real estate is “free”. However, if this land were to be reclaimed, it would have a market value, and that unrealised value should be considered a cost against Option F. There is nothing to stop the government allowing reclamation for other uses of course and pocketing the gains from the change in land use. So there is an opportunity cost in using this land for a motorway.
Talking about cost vs benefits of a highway.
In the city we have Britomart, where a lot of people enter the city on trains and buses. And we have the ends of Fanshawe Street, Nelson Street and Cook Street, where a lot of people enter the city in cars. Now, in my opinion the first of these is definitely the nicer place to hang out.
How is that accounted for in those cost/benefit models?
What hope do we have of contacting the two people who can stop this? Bill English, and the new CEO, Fergus Gamie?
Ah.. But option F is probably not being compared incrementally to option B. Instead if you compare it incrementally to Option A you get incremental benefits of 750m with incremental costs of 600m resulting in an incremental BCR of 1.25. The incremental BCR of 2.7 for option B is still a lot better, but it is now much easier to justify picking F over B if there are other reasons for preferring it.
What may be useful is to compare the NPV.
The NPV of Option A is 650-700m
The NPV for Option B is an additional 500m for a total of 1150-1200m
The NPV for Option F is an additional 75m for a total of only 725-800m
Option B yields an additional NPV that is 6.67 times that for Option F
“Ah.. But option F is probably not being compared incrementally to option B.”
So they are incompetent? Should we let them know they have made such a basic error in their assessment?
It seems to me that the BCR is ex post facto. The trucking companies have made their wishes clear and, from what I can see, the strategy is to give them what they want. The “strategic fit” gambit will render the relative BCRs of the various options irrelevant. That being said, qualitatively how do any of these options give rise to agglomeration benefits? Photo of a $50 note for the best answer.
By allowing more vehicles to move through the corridor at a more reliable pace more businesses are encouraged to locate in the area. This gives agglomeration benefits in improved productivity and knowledge pooling in the area.
I wonder if NZTA or AT are noticing what’s happening to Onehunga landuse beyond the MetroPort operation? It seems to me that the ASKL wide dwelling supply gap is exerting itself on the commercial areas there and they are increasing conversion from commercial use to residential are occurring? The area has good bones, the distance to the city is great, spillover effects from fully priced existing dorm burbs to the north; gentrification here is all but inevitable
A mega truck super highway may not be the smartest thing that this area needs; industry looks like it is starting to migrate south….?
“By allowing more vehicles to move through the corridor at a more reliable pace more businesses are encouraged to locate in the area.”
So these new roads are going to reduce congestion (really?) and businesses are going to locate there to take advantage of this improvement. Are they completely new businesses or are they relocating? If the former are the agglomeration benefits greater than if they located in another area? If the latter should we not deduct the agglomeration disbenefits from leaving their previous location?
“This gives agglomeration benefits in improved productivity”
Are you suggesting that these roading proposals will result in a national net higher density of businesses and hence a higher national net productivity?
“and knowledge pooling in the area.”
Really? I shall assume (reluctantly) that in this instance such knowledge pooling is significant and increases productivity. Why would this not occur elsewhere absent this road project?
Here’s a hypothesis to ponder: A business that moves to the area in the belief that the road project will lead to reduced congestion is too stupid to be a highly productive business.
@ Sailor Boy +1 good point.
Is there room within the CBR for another factor such as the induced demand for the next project where the proposed project ends and congestion at the end of the project creates demand for another project?