All the way back in June 2014, Matt asked the Ministry of Transport if he could see some of the background research that informed the 2015/2016 Government Policy Statement on Land Transport Funding. They were slow in responding to his Official Information Act request, but the information has gradually trickled out. Two days before Christmas, MoT sent him a long, very interesting working paper on New Zealand’s capital spending on roads.
Over the next four weeks, I’m going to review some of the key findings from the paper. To briefly summarise: there are big issues with the land transport budget. The paper suggests that current road spending may represent a massive misallocation of capital. To their credit, MoT appear to be acknowledging this. However, it doesn’t seem to have percolated up into the investment decisions being made by the Government.
The first things that jump out from the report are the demand forecasts being used to predict the need for more roads. As we have been observing for years, overall vehicle kilometres travelled (VKT) in New Zealand have been flat for almost a decade. (In line with trends throughout the developed world and in almost every US state.) Here are MoT’s estimates of VKT on state highways (yellow line) and local roads (green line) since 2001:
It is reasonable to assume that the eight-year-old trend of flat traffic volumes will continue, at least in the near term. But MoT’s paper suggests that the government’s not making that assumption. They commissioned consultancy NZIER to forecast future traffic growth over the period covered by the 2015/16 GPS. Here are their results:
These forecasts are a bit laughable. The paper offers no explanation for why we should expect the trend of the last eight years to immediately reverse itself. Fortunately, MoT seems to be aware that NZIER’s forecasts are unreliable. In order to demonstrate that, they ran NZIER’s forecasting model backwards, using data from previous years. They found that the model did not match past data very well – often a sign of a poorly-designed model:
As we discussed last year, since 2006 MoT’s own forecasts for future VKT have all proven too optimistic in the long run. However, unlike NZIER’s model, they have often picked short-term trends accurately:
[Note: The paper contains a number of other charts summarising various forecasts – I’ve only excerpted a few of the more interesting ones. It’s well worth taking a look at the rest.]
What does all this mean? I had two very different reactions – one technical and one policy-focused.
First, from a technical perspective, what is going on with NZIER’s forecasts? I’m not familiar with their models, other than the vague statement in the MoT paper that they were based on projections for population and economic growth. However, it seems like they may be based on careless use of regression models, including:
- Omitted variable bias – it seems like NZIER may have assumed that traffic grows in proportion to one or two variables – e.g. population and economic growth. In doing so, they may have excluded other important variables, such as the structure of the economy, the location of projected population growth, and the impact of technology (e.g. online shopping, smartphones, video calling).
- Model mis-specification – NZIER may have assumed a positive, linear relationship between (say) income levels and per-capita VKT. However, the actual relationship may be more complicated. It might actually be true that people drive more until they reach a certain income level, and then start driving less.
- Inappropriate assumptions about causality – Regression models measure the degree to which two variables are correlated. Establishing that there is a causal relationship is much more difficult. NZIER may have (wrongly) assumed that a past relationship must necessarily hold true in the future. And they probably haven’t explicitly accounted for the impact of investment decisions in shaping demand.
But that’s all just economic mumbo-jumbo. Ultimately, I’m more worried about the implications of these forecasts for investment decisions. Are bad forecasts being used to justify a road-heavy investment programme? If NZIER’s forecasts are being sensibly discounted (and ignored) by transport agencies, I’m not that worried about them. Economists (myself included) make ludicrous statements on a regular basis, usually to little effect.
But if they are being taken seriously, New Zealand’s taxpayers should be concerned. If policymakers and planners are basing major investment decisions on the NZIER forecasts, they may misallocate scarce funds to roads that will not be needed. That is money that could be better spent on other things, such as bringing New Zealand’s underdeveloped public transport and walking and cycling options up to scratch, building new schools and hospitals, or rebuilding Christchurch.
Next week: What’s happened to BCRs for new state highway projects?