The new Government Policy Statement sets a bold and far more balanced policy direction for transport in New Zealand than we have seen for decades. Its strategic direction is based around four key objectives:

As well as setting the balance of funding across different areas, the other important thing the GPS does is identify the policy priorities that NZTA must give effect to through their funding decisions.

For NZTA to do this, they create a more detailed document called the “Investment Assessment Framework” (IAF). The document itself is a bit of a mind-numbingly boring read filled with jargon, but at its heart the IAF is the “GPS in action” and ultimately is the document that determines whether NZTA funding will go to one project or another. The diagram below shows how the IAF relates to other transport planning documents:

Previous versions of the IAF ranked projects based on a combination of Strategic Fit (how well aligned the project was with government priorities), Effectiveness (the extent to which the project solved the problem being faced) and efficiency (the cost-benefit ratio). This time around that has been simplified to just results alignment (essentially a combination of Strategic Fit and Effectiveness) and a cost-benefit analysis. The important “ranking table” is shown below:

Broadly, the ranking approach seems to look first for projects that are “very high” for either results alignment or cost-benefit ratio, before reverting to a typical “high/high”, “high/medium and medium/high” ranking process. This does create some strange scenarios where projects that might perform poorly on one of these will still be ranked quite high. Given the major issues we have with transport cost-benefit analysis processes, the potential for a misaligned project to still get funding because it appears to have a high BCR is a risk to keep an eye out for.

One useful element of the new IAF relates to how the GPS’s greater focus on safety is being turned into a reality. There are countless stories of how critical safety projects get knocked back because they create small delays for traffic, and it seems like the IAF is taking some first steps towards ensuring that does not continue to happen:

Ultimately it still seems stupid that safety gets traded off against other transport outcomes in this process. This doesn’t happen in other sectors – health and safety is seen as a critical requirement that must be met, and then other goals are considered. This baby step is certainly still not consistent with Vision Zero thinking, but it’s a start.

The heart of the IAF is the guidance for how “results alignment” is determined in the prioritisation process. This is done for each of the main investment types (public transport, safety, new roads etc.) The detail is a little bit tedious (because ultimately it does guide where millions of dollars is or is not spent) but ultimately you can see how things like the proposed rapid transit routes in ATAP would get a “very high” results alignment:

With walking and cycling you can also see how much of the proposed cycling network would also rank “very high”, especially critical missing links like SkyPath:

Overall the IAF is a useful step in the right direction, but one can’t help but feel it’s the bare minimum level of change to give effect to the GPS. In areas like safety there’s still a way to go, while also there’s a need to ensure the balance between results alignment and the cost-benefit analysis is correct. It would be great to be able to just use the cost-benefit analysis process, because it would do away with all the complex qualitative judgements in results alignment. But until the process for undertaking transport CBAs is improved, I guess we’re stuck with this hybrid approach.

Share this

39 comments

  1. Thanks, Matt. Important stuff. What I want to know is if anything in the IAF will actually help reshape the ATAP and RLTP so that they don’t include all those traffic-inducing projects that increase road capacity.

    The Botany Rd’s intersection expansion project, for example, is completely out of line with the Sustainability Framework and the GPS. Only because of the imbalance in the ATAP and RLTP does something as harmful as that project somehow get through.

    I know the process for evaluating projects is utterly broken, with NZTA refusing to acknowledge the new trips induced by roading capacity increases, but by giving the Botany Rd intersection expansion project the low ratings it deserves, could it – under the IAF – have been excluded from ATAP anyway?

  2. It will be interesting what the IAF will say about Greater Christchurch getting a rapid transit network? The cost benefit ratio and the safety improvements should be pretty good given it is an urban area the size of South Auckland with no rapid transit services.
    Here is the proposal I think has the biggest benefits for the smallest outlay.
    https://medium.com/land-buildings-identity-and-values/ending-christchurchs-car-dependency-culture-can-help-revive-the-city-332f6786baa

    1. Just had a read, interesting proposal. My gut feel is the BCR for rapid transit in Chch would be quite low, while the strategic fit would be high.

      The only rail line that is ready to go with minimal investment is the line to Rolleston and that has a very poor catchment at the moment with a lot of industry around the line. The line to Rangiora has better catchment but needs more investment, namely passing loops and ideally realigning the tracks at Addington to allow a straight run into the city.

      I don’t think the station location in the city is a killer, many Wellington commuters have a decent walk. Also the centre of business appears to be moving south anyway, the two biggest office blocks on Armagh St (PWC and Forsyth Barr) are both gone, and a lot of construction is now happening south of Cathedral Square.

      Investing in rapid transit in Chch is a good idea, but it is definitely a long game, much like it was in Auckland in the late 90s.

      1. Upgrading the tracks shouldn’t be too expensive. Both Auckland and Wellington have had significant double tracking and electrification in the last decade or so as a matter of course

        The proposed bus tunnel and new Moorhouse Ave train station would be a significant investment but it would make a central bus rapid transit corridor viable, which would be hugely beneficial in making the whole bus network more efficient.

        The tunnel and train atation would be Christchurch’s version of Britomart and the Northern Expressway infrastructures.

        I think the BCR for rapid transit in Christchurch would be better than Wellington because rapid transit in Christchurch is a paradigm shift not a small evolution of an existing system. Also Greater Christchurch has faster population growth to tap into.

        1. I agree it won’t be expensive relative to many other projects. However, I think the benefits will be quite small to start out with as I don’t think patronage would be that high to start with, there would have to be land use change.

          For example the Rolleston line would likely have three suburban stops – Rolleston, Templeton and Hornby. Rolleston is about half the size of Pukekohe, Templeton is tiny and Hornby would have a walk-up catchment smaller than most stations in Auckland. This will likely require is large operational subsidy.

          I fully support rail in Chch, however we need to be realistic – it is going to be a long game to build patronage and it will require some brave politicians.

          1. I think the northern line would initially be more popular than the western line. There are larger catchments in Mona Vale, Papanui, Belfast, Kaiapoi and Rangiora. Also there are more road congestion pinch points to the north.

            The BRT central corridor would be a fast right of way and should drive an increase in bus patronage because it would service all the routes in the northern and southern suburbs of Christchurch plus satellite towns like Woodend and Lincoln which are not on train lines.

            Christchurch is building about 4000 houses a year -so land use changes over time will be quite significant.

  3. On the safety being weighed against other transport outcomes, another thing to throw into the mix is that Ellison has committed AT to this recommendation of the Road Safety Business Review:

    “Remove the NZTA requirement for calculation of crash reduction benefits for proposed
    infrastructure safety treatements to be offset against the value of time costs due to delays.”

  4. Same old. Same old.

    Surely one way to fix this is to alter the BC criteria so that safety impacts/benefits are assessed in the BC just like other supposedly “nebulous” concepts such as Wider Economic Benefits (WEBs) are?

    That way, a project which has huge or even medium safety benefits will score highly on the BC analysis. Even if the alignment is not high, so it should still get a look in.

    But you are right, the normal process is we would expect NZTA to only consider projects in the first place that (a) do not make things worse safety-wise and (b) improve on safety where that is relevant. Otherwise we will never actually get anywhere.

    So much of our roading system is more akin to “safety last” – we build safety “in” – only if nothing else is impacted by doing so, or if it doesn’t add much to the costs.

    NZTA will still however justify some extraordinarily expensive projects on “safety” grounds though if it suits their purposes.

    1. Yes, and a growing number of externalities are being studied to the point of being able to put in dollar values for them in the business cases. For example, we’ve long been able to put values in for the environmental benefits of reducing the number of lanes on a road, but now the research gives us values for severance. (The benefits of reducing the number of road lanes are just above £1 per walking trip. The benefits of reducing high traffic levels are also above £1 per walking trip.)

    2. I’m not sure I follow: Arguably safety benefits are already included in benefit-cost analysis?

      If there’s a sense they are too low (this is something I share), then what we need is logical arguments why the assumed values for safety benefits are too low.

      My take, for what it’s worth, draws on findings related to imperfect information and behavioural economics. That is, many people don’t have the information they need to accurately assess risks and effects of accidents. Plus, for low probability events there’s evidence that people round up and down: sometimes to zero. In the latter case, many people assume the probabilities of accidents are zero, when we know (objectively) they’re anything but.

      1. So you don’t think there’s anything to Matt’s observation that “in other sectors – health and safety is seen as a critical requirement that must be met, and then other goals are considered.” Should safety have to be enumerated sufficiently to win over a business case analysis, or should it be a pre-requisite to the business case?

      2. I’ve got a slightly different critique. Existing methods for ex-ante estimation of changes in crash rates resulting from improvements rest upon some statistical assumptions that are likely to be violated in practice, especially for low-n events such as pedestrian and cyclist crashes.

        In particular, safety assessments often rely upon an analysis of five years’ worth of observed crash data. Because pedestrian and cyclist crashes resulting in death are fairly infrequent – a few dozen a year for the entire country – this method is simply not adequate to identify which intersections are safe or unsafe. (This is easy to prove with Monte Carlo simulation!)

        There’s actually an even worse problem lurking in the statistical weeds. Ex-ante safety assessments are likely to be fundamentally flawed, but ex-post assessments may not be any better.

        Conventional statistical tests (eg t-tests) will tend to under-estimate the statistical significance of big changes in the likelihood of low-frequency events even in comparatively large samples. Even if you use a method that makes a more plausible assumption about the distribution of low-frequency crashes (eg Poisson distribution), you still need a rather large sample to get meaningful results. (Again, easy to demonstrate with Monte Carlo simulation.)

        My suspicion is that this creates a *systematic* bias against identifying and prioritising safety improvements for pedestrians and cyclists, and potentially a bias *towards* road designs that endanger people on foot or bike. This bias can arise even when everyone has the best evidence-based intentions.

        To correct it, we would have to take a different approach to identifying which intersections are unsafe – for instance, by doing observational studies of near misses or interviewing people about perceived safety – and counting the benefits of improving those intersections.

        1. The problem with that approach is that a site that is perceived as unsafe can have a better outcome, as people take a lot more care. The real problems occur when people perceive something as safe when it isn’t. That use to happen at zebra crossings over four lane roads. Everyone thought they were safe but in reality nobody was.

          1. Excellent point. This would tend to further undermine any attempts to divine whether sites are safe or unsafe based on observed crash records.

            Hmmm, there’s probably a good paper in this…

          2. I’ve thought exactly the same, Peter, but was hoping someone had already done it. The ‘research’ behind decisions to remove pedestrian crossings always flashed warning signs to me as lacking analysis of what the pedestrians were doing instead – crossing elsewhere (possibly highlighting a little-known infrastructure improvement somewhere, and possibly just risking their lives), or not crossing, with the economic, opportunity and low physical activity consequences of that.

          3. I think when you understand there are problems with your data then you are already half way there. You don’t give up on data but you change how you use it. One of the best data sets we had was the crash monitoring data that Collette Kraus used to compile for the MoT. It took all the crash reduction studies and the types of mitigation and the dates they were implemented. She then did before and after studies based on many sites. The problem with crash reduction studies is sites are selected if they have a lot of crashes. That can be because they are unsafe, or it can be because the variance is high compared to the mean when you are dealing with small numbers. Regression to the mean suggests you will see a reduction in some sites even if you do nothing. But at least the monitoring programme avoided the worst of that by looking at many sites and using a longer time frame. My long winded point is biased data is ok if you know how it is biased.

        2. This seems true to me.

          For all that the raw numbers of DSIs are high, the sample is actually quite small relative to the network and normal activities and how harm manifests from a given environment of risk.

          It is possible to see significant reductions in unsafe driver behaviours, leading to literally thousands of dollars a month in reduced operating costs for even moderately sized fleets (safely driven heavy vehicles tend to send more time running within optimal mechanical parameters), yet the impact is still invisible in terms of changes in DSIs because the conversion rate from risky driving to actual harm is so low.

          It is getting more and more possible to assess the physical characteristics of corners and intersections and their contribution to risk, but there seems to remain the big dependency on behaviour and the credibility of enforcement of the rules at these points.

        3. Peter, I agree, and safety engineers have developed such methods of identifying safe and unsafe infrastructure. Miffy’s concern may be valid, but we can work with that, too; crash data can surely be a check back on the validity of these other tools.

          My question remains for it all: where in the IAF is the assessment process to ensure that all the data collection and safety work gets the funding it requires?

          NZTA apparently did pedestrian counts for the Waterview Connection. They lost them, though. I think we can correctly infer from this that the ongoing use of the data was given no importance within the agency.

          1. Traffic Engineers have long argued that various terrifyingly dangerous intersections for pedestrians are actually proven to be safe, when relatively few pedestrians are recorded as dying at these often high traffic volume sites. But that can simply be explained by pedestrians largely avoid these places, as they generally wish to remain alive. Additionally, as Miffy observes above, they may also behave extremely cautiously if forced to use them.

            However this in no way makes for successful design, simply frightening away a class of users should use a huge negative number on the BRC, but of course, it doesn’t.

            An example of this is TEs arguing that pedestrian crossings on slip lanes are dangerous, because drivers don’t stop, and pedestrians walk as they have the right of way. Rather than observing that the whole idea of slip lanes is inherently lethal outside of fully grade separate systems. Or indeed that we are clearly not enforcing pedestrian crossing violations sufficiently across the system to change their use by drivers.

            Miffy is right about ‘safety through uncertainty’, but this only works when all users are uncertain, and calmed an watchful. Nowhere in this doctrine is the idea of expediting one dangerous set of road users, drivers, and terrifying the rest into fearful scuttling. Yet that is often what we have.

          2. I have seen plenty of examples where people have argued no provision for pedestrians is better than an unsafe facility. My view is that is and was always a mistake. That also happened with the example I gave of zebra crossings on four lane roads. Rather than signalising every crossing, many were removed which left kids having to run across busy roads to get to school in poor areas while in wealthy areas parents would drive kids.
            But my point is that we can’t always predict what will be safe based on whether it complies with someone’s standard. We need to use data even though we know the data contains bias.
            Two examples I have are when the late Russell Dickson showed me that the Cook Street off ramp, which has a substandard exit taper and the Nelson St off ramp which was a right lane exit, both had lower crash rates than typical off ramps despite both falling well short of design standards.
            The lesson is you need humility when you try and predict the future and you need as much data as you can get. If the data is biased then understand how it is biased and use it knowing that and adjust your view accordingly.

          3. It’s hard to get data of how many people aren’t using a place cos it’s scary, until that is it is fixed…

          4. Miffy, were you able to see the full set of crash data for the Nelson St offramp, ie were there crashes that occurred in the lead-up to the ramp, as people suddenly realised they were in the wrong lane? And after it, as they frustratedly realised they’d missed the exit, and were a bit confused as to what to do?

          5. Partially answering Heidi’s question to Miffy regarding the former Nelson Street off ramp right hand exit. As a local I think it would be difficult to discriminate whether nose to tails could be attributed specifically to manoeuvres related to that strange off ramp location against the now extremly compressed three way City, Western Motorway and Nelson Street off ramp locations. My feeling, from our own visitors arriving from the south, was that a lot experienced a double harbour bridge crossing, (one at least, a quadriple harbour crossing) having missed that turnoff, but then they reduced their chances of a pile up under the current unsafe overly compressed configuration.

          6. Russell showed me his analysis over a cup of tea while we were doing a safety audit for another job. He used a good length in advance for both off ramps and the other ramps he used for comparison. But as you suggest I think the number of u-turns over at the Onewa interchange decreased significantly when the Nelson St off ramp was ‘fixed’.
            It is easy to forget that this stuff we do isn’t science. It is a social science where any change you make needs to take into account how people react. Not enough of that is taught.

        4. In Health and Safety this is a known issue and is dealt with by focusing on reducing near misses and minor injuries. It is found by reducing these, the number of medium and serious injuries and deaths are reduced.

          Sadly when Engineers go and work out they conform to an organisations H&S policy, but find it difficult to apply it in their planning. Which is why there is a focus on reduction of deaths and serious injury but no idea where to apply a treatment, rather than the more effective reduction of near misses and minor injuries.

          1. Focusing on reducing near misses and minor injuries is one strand to safety work. Non-motorised user audits is another.

  5. I could not see in the IAF, how road controlling authorities will be funded to gather important pedestrian volume data. They need to work together to gather the “denominator” in the incidents per pedestrian crossing equation, and data about preferences of pedestrians for different infrastructure types. Compared to all the data available feeding safety improvements for car occupants, this data is sorely lacking.

    Can anyone see how data collection such as this will be assess for funding under the IAF?

    1. I was under the impression that it was usually picked up through the planning and investment activity class, or whatever that is called now. The NZTA has the right and means to support more detailed data gathering as projects work their way through the various layers of business case. So the data gathering is not assessed independently of the project or program to which it relates.

      Conceivably network level studies for local planning purposes could also attract funding, again through the same general activity class???

      There is also a more pure research stream of funding available. That tended to be allocated with an eye to the priorities of day; presumably bulk buying the surveys as part of a meta-study, for example, might be able to attract funding that way…

      1. Sorry, just saw this comment. Thanks. So do you think the Minister could direct the NZTA to collect the data as part of each project?

        1. I think:

          – ex ante, gathering the data is a necessary consequence of valuing the impact/outcome, so the NZTA is obligated to develop the evidence base by its statutory duties.

          – ex post, the Minister has set the terms in the monitoring and reporting parts of GPS 2018. The (perennial) struggle for MOT is to lever that into useful data gathering, evaluation and reporting ‘lower down’.

          For the most part monitoring the GPS gets snagged at the general statements made via the NZTA Annual Reports: little success seems to have been enjoyed in getting deeper/finer-grained reporting happening below that level of aggregation or with any meaningful level of benchmarking. And don’t get me started on the ‘value’ of those ever-shifting metrics…

          In theory the NZTA establishes the primary outcomes a project is expected to deliver on, based on its ex ante evidence gathering and modelling. Ex post data-gathering aligned to this subset is meant to occur if the project is selected for evaluation, which is meant to be certain for all big projects (I can’t remember the threshold, sorry) and a solid percentage of others. In each case it is scaled to project size and cost.

          So, as the investment logic and weighting of outcomes/benefits shift to align to the new priorities, the accompanying data-gathering, both ex ante and ex post, should also shift.

          I don’t recall the work on either GPS 2015 or GPS 2018 being able to tell an aggregated evaluation story – Economissive would have put it into his work if it existed. Although, tbf, we are really only now at a point in time where you could evaluate the impact of GPS 2009 and GPS 2012.

          The big change that is needed, I think, is that the Minister (any Transport Minister) – and probably Parliament – need to ask for the evidence of performance (results ex post vs ex ante benchmarks and estimates), not just the gathering of data. Create demand at the back end of the system to change behaviour through the whole…

  6. Two questions?
    – Are Greater Auckland included in the process as an Approved Organisation or Other Stakeholder?
    – Will projects with a BCR under 1 remain unfunded, or will they be removed form the project list?

  7. There is no absolutely safety. People can be striked by lightning. A meteor can strike you.
    Absolute safety is doing nothing. Train running at 0kmh is absolute safe.

    Therefore there is a trade off.

  8. On BCRs, forecasts, and transport infra the case of Swedish city Malmo and its City Tunnel project is interesting.

    Opened in 2010 it is a 17km underground rail tunnel connecting the previously terminating central station to the Oresund bridge to Denmark. Enabling through-routing services with a new mid-city station and another in a more edge-city development zone.

    On the financial projections and outcomes: From here: https://en.wikipedia.org/wiki/City_Tunnel_(Malmö)

    ‘According to an economic analysis conducted at the Center for Transport Economics (CTEK), the tunnel will produce a result ranging from slightly negative to slightly positive; most likely, it will bear its own costs. According to the Swedish Transport Administration’s own forecasts, the project will not be profitable. It should be noted that these calculations were carried out in 2001, and since the early 2000s there has been a sharp increase in local and regional train journeys, making calculations from 2001 highly unreliable. In April 2008 the Swedish Transport Administration revised its forecasts, stating that “…the economic viability of many railway projects in the real world will exceed the previously estimated [sic].”[3] The report was commissioned as part of the basis for infrastructure investment in the transport sector from 2010 to 2021. According to a political principle established in Sweden during recent years, construction projects in areas where construction is overly expensive does not need to be fully economically viable. In such cases, a lower cost estimate can be used in the calculation, since it is not politically desirable for certain metropolitan areas to be neglected merely because of the high cost associated with construction in these areas. Without such a principle, it would not have been possible to build the City Tunnel, Södra länken, Götatunneln and other upcoming city projects.’

Leave a Reply