Two weeks ago I did a post on the original business case for the Northern Busway which showed it had a benefit cost ratio (BCR) of 1.2 and I also pointed out that things like patronage had greatly exceeded what was expected so the BCR was likely well under estimated. As part of obtaining that business case I also found out that the NZTA had commissioned a post implementation review of the busway and that review has now been completed (although they have pointed out that the views in the report are those of the consultant and not the NZTA), there are two parts, the report (800 KB) and the annexes (2 MB).
There where three key goals for the review
- To review public transport (PT) economic benefits and procedures, including those specified in the EEM. Develop an overview of current public transport evaluation in NZ and provide a context for the post-implementation review (PIR) of the Northern Busway (NB).
- Undertake a PIR of the NB in terms of costs, benefits and wider outcomes. Quantify the range of benefits delivered by the NB using available data and appropriate methodologies.
- Discuss implications for current NZ PT evaluation procedures. Determine if any identified benefits from the NB are unaccounted for in current NZ PT evaluation procedures and make recommendations accordingly, taking account of: the wider PT evaluation review, the NB PIR and alternative approaches to cost benefit analysis.
I am going to focus on number 2 first as that specifically relates to the business case of the busway. The original business case was completed in 2004 and here are some of the key differneces between what was forecast and what actually happened.
The consultants have then taken the information and fed it through a number of different economic models to produce different BCRs, the first set looks at benefits from purely an infrastructure view:
The total variation on the BCR range of between 1.2 and 5.2 may initially seem surprising, but this illustrates just how important the core assumptions adopted when undertaking the evaluation are. For example, the highest BCR reduces to 2.6 if the tests relating to discount rate and evaluation period are excluded.
The longer evaluation period (of 60 years compared with the EEM 30 year period) reflects the fact that major infrastructure is constructed as a long term investment.
The original evaluation BCR of 1.2 represents a very conservative approach covering a narrow benefit range, whilst the highest BCR of 5.2 is closer to international practice and represents a more comprehensive benefit range.
There was also an experimental model built to look at things from a total package perspective which includes not only the infrastructure but also the running facilities and services, the outcome is below:
What is pretty clear is that when they were first assessed the projects they didn’t really paint the full picture of benefits that would be achieved by the project. Some of that is due to to an older methodology but even the current version seems to have some gaping holes in in when it comes to public transport which is highlighted in other parts of the report and the recommendations. One of the most concerning issues is with the application of how travel times are calculated. The report has this to say about it:
In the evaluation of transport projects, travel time savings typically represent a very large proportion of total benefits and so deserve particular attention.
For the NB, there are some particular aspects concerning
For the NB, there are some particular aspects concerning the application of travel time values that require consideration as follows:
- Users of the busway have high car availability, with up to 50% of new users being former car drivers and even more who have a car available
- The EEM treats existing¹ public transport commuters as having a significantly lower value of time than car drivers. In addition, all car drivers and car passengers have their time values increased in congested conditions. Only those who are standing on PT (a small minority) have their time increased, but even this this remains well short of car time values. The contrast between car and bus is even greater for non-work travel purposes.
- Whilst this approach may be justified for some PT evaluation purposes, a different approach is required in the case of rapid transit system (such as NB) users because of the high service quality and significant mode choice available. In such circumstances it is illogical to assume that NB users have a lower value of time than car users.
- There is a strong case in the case of the NB, for PT user time values to be at least equal to average car (driver and passenger) time values. In other words, the average bus user time value (whether seated or standing) are likely to be at least equivalent to the value of car users.
- The approach of using an equity value of time for all modes is applied within evaluation procedures elsewhere, for example in the UK. This is particularly important to avoid potential distortions when evaluating rapid transit systems (such as the NB) which have a high proportion of car available users.
¹The EEM now allows new users who were formerly car drivers to retain their higher value of time when using PT.
When it comes to how PT projects are assessed it seems that the main conclusions we can take from it are:
- NZTA’s current method of calculation BCRs for PT projects are pretty undeveloped and in many respects inconsistent with overseas best practice
- In particular, major PT projects have very long-standing benefits and longer evaluation periods & lower discount rates seem particularly appropriate
- It’s likely that the original business case for the Northern Busway under-estimated its benefits, though the extent to which it did so is a bit difficult to tell as it was assessed in a pretty suboptimal way at the time.
- NZTA have some pretty major work to do in order to bring their assessment of PT projects in line with international best practice
Hopefully some of these lessons will be able to be incorporated into other major PT projects, especially a certain tunnel.