A few months ago we learned just how fragile our train system is, even after re signalling it when the the whole system was brought to its knees due to a faulty power supply that that fed the desk from which the Auckland network is managed from. Last night something similar happened once again with some sort of signal fault once again taking out the entire Auckland network for a period of time during the rush hour.

Auckland train services are starting to move again after hundreds of Auckland train commuters were left stranded at Britomart Station.

An announcement was made about 6.25pm that no trains were running due to a signal failure in Auckland. The failure was originally reported as being in Wellington.

By 7pm, Auckland Transport said signals were now operational, but passengers should still expect “significant delays”.

It was originally reported as being another fault in Wellington however Kiwirail is now saying the fault occurred in Auckland. Regardless of where the fault originated it is pretty shocking that we have now had two faults that have taken out the entire network at such a critical time, especially considering the system is virtually brand new. This kind of problem is clearly not a good look and just last month at the Auckland Transport board meeting staff promised that we wouldn’t see any repeat. My guess is there will be some sort of investigation that occurs where there are a whole pile of fingers pointed and some small changes will be made but once again no one will really be held accountable.

In may ways it is a real shame as it contributes to taking the gloss off what is meant to be a pretty high quality and advanced system even compared to many other places around world. What has been installed is a European Train Control System Level 1 (ETCS Level 1) although some parts of it aren’t even active yet and are waiting on the arrival of our new trains to take advantage of the new features. One thing that has gone live recently is an automatic signalling where a computer controls most of the mundane tasks of running the network freeing up staff to deal with the more complex issues. It is the first time that the system has been used in New Zealand and here is a bit of an explanation as to how it works:

Automatic Route Setting works within the following way: a computer database holds the master timetable for the Auckland network. The data includes train numbers, train types, all stop times, platform allocations and routes.

Operational Planners then programme preset “commands” and “check conditions” to set points and clear signals.

These commands are saved as “triggers” and associated to a particular location on the track. When a train is detected at that track location, the computer identifies the train number, and therefore its route.

The logic then looks at the “check conditions” (such as making sure other trains have passed or the train crew have operated a train-ready device) and if they comply, it will clear the relevant signals for the correct path. All this happens in seconds.

If the conditions for a particular command are not met, then the system will retry for up to five minutes before alerting the Train Controller that the route will not clear.

Over time, more and more trigger locations will be set throughout the network.

For example, a southbound train arriving at Ellerslie will trigger a command to set the route at Penrose to go to or from Westfield. Those signals will be set by the computer if it sees no other trains in the way.

If there is another train present, the train will trigger another request when it reached a second trigger closer to Penrose Junction. If there is still another train in the way, that command will keep trying until the track is
free or five minutes expires.

This means the train is getting the correct route as quickly as possible after the track is free.

Share this

41 comments

  1. Corners cut during installation or customisation done against the manufacturer’s reccomendation not to. Lets see what the investigation turns up.

    As for finger pointing, it would be nice to see this stop and one or two individuals take responsibility for actually fixing the problems and ensuring goong forward that they dont happen again.

      1. An independent audit of the setup / system should be done – by people not directed connected to the equipment suppliers, Kiwirail, AT or NZTA. Suggest a company from outside the Australia / NZ region. I know of such a company in Japan that well qualified to do this if AT would like to contact me about bringing them in.

        1. Apologies- some typos and grammatical errors thus I shall state this again:

          An independent audit of the setup / system should be done – by people not connected to the equipment suppliers, Kiwirail, AT or NZTA in any way. Suggest a company from outside the Australia / NZ region. I know of such a company in Japan that is well qualified to undertake the task if AT would like to contact me about bringing them in.

  2. Again, I say Auckland Transport should compensate affected travellers with a Hop credit ($10?) when delays happen. It wouldn’t be hard to identify all those that tagged on during the affected period, and apply the credit programmatically. On charge it to KiwiRail. There has been a lot of focus on the new PTOM operating model for buses, but I’d like to see the rail contract get some teeth for customers as well.

    1. In Melbourne if they have a particularly bad network failure or a spate of isolated failures in rapid succession, they declare a free travel day and open the gates up as compensation.

  3. “taking the gloss off” is being very kind.
    I’m sitting on a train at Britomart that is parked up, with no indication of when it might depart. it’s chaos here. 34 staff on the platforms and no idea what’s going on. Is it not possible to have an electronic
    network map

      1. Hi Rob – My reply was cut off (smartphone), sorry, but I meant something like an electronic map (even web-based) showing the locations of each train in real time, with estimated arrival times, etc. I didn’t think that would be too difficult to implement and would ease a lot of frustration. With wait times >1hr (and regular delays of >15mins) it’s important to know if you should cancel business meetings, order a taxi , etc – even grabbing a coffee or meal is a better use of time than waiting on a platform, indefinitely.

  4. At least in the UK, public transport failures were an acceptable reason for being late to work. Obviously if they happen to often the employer will expect the employee to change transport modes.

    I realise that’s not a lot of compensation to passengers stuck in the trains. And I agree that a hop credit is a great idea for people affected by equipment failures.

  5. The announcement may have been made at 6:25pm but the 5:53 train I was on ground to a halt at the exit of the Britomart tunnel around 5:55pm and didn’t start rolling again until 6:44pm. During this time there were exactly two announcements over the PA both blaming ” an operational incident” (which, by the way, is a phrase that should be banned; they are effectively saying we are stopped because something has made us stop) and claiming that the on board staff were working to find out how long we would be delayed. In fact, the on board staff – one on duty, I think, and three off duty ones heading home – were huddled in one corner of my carriage rebuffing all attempts to ask them for information.

    Two passengers in my carriage took turns phoning the AT Help Line who were very helpful and kept us up to date predicting – to the minute – when we would be moving again. Its just a shame that the staff didn’t think to do this.

    Oh, and then this morning there was a broken down train just after New Lynn and we were half an hour late again (nothing to do with the Southern Line problems).

    1. Good god. Yet another example of subbies (HILOR is it??) excavating before checking the exact location of cables. Telecom, Watercare, Mercury, AT…the regular occurrence and subsequent acceptance of these incidents has got to stop The contractor needs to be properly fined and made to issue a public apology. Only then will everyone involved learn that finger pointing, arse-covering and overall acceptance of such modus operandi will no longer be tolerated.

  6. What does a signal fault actually mean? Does it mean the traffic lights aren’t working? Isn’t there a manual backup system?

      1. I assume the fibre optic cable is what is sending all the information to train control about where the trains are and what state etc the signals are in. So with that cut (and I’m just guessing here) the TC is effectively blind…

        1. Conduct an independent audit of the system – by people from outside the AUST/NZ region who are not connected in any way to AT, Kiwirail, NZTA nor any of the companies subbing to the aforementioned.

        2. A duplicate cable likely means redundant switches or routers, and a whole lot of functionality to cut over from one set of components to the other in the event of network failures. That means extra expense for the initial deployment, obviously. But it’ll be a more complex infrastructure that is harder to understand, test, and make changes to. The extra complexity may result in the system being less stable than it would be if it were simpler. Any decision to go with single or redundant infrastructure would have to take in to account the expected failure rates and the consequences of failure… but in my experience, simple systems have a lot of merit.

        3. If you have duplicate systems for 99.9% of your equipment, you can guarantee that the digger will hit the 0.1% that doesn’t.

        4. How did trains work in the 1800s without fibre optic cables?

          Surely their should be a manual backup system. Like send some people out to the signalling sensor places with mobile phones and stop/go signs.

        5. “How did trains work in the 1800s without fibre optic cables?”
          With a signal box every few kilometers (so maybe thirty for the Auckland area), each requiring three plus signallers to cover the work 24/7 : maybe a hundred staff – but such labour was relatively cheap then, so there was no real need to invest in researching more efficient technology..

  7. Many of the people arriving at Britomart yesterday evening during the outage didn’t find out about the problem until they’d tagged through the gates with AT HOP cards. Then incurred charges to get out of the paid area, on top of having to find (and pay for) alternative transport

    1. This simply isn’t true, I arrived at britomart at 620, at the Galway street end, and was told by the staff member on the gate that the had been a fault, and that the trains were not running, this was around 10 minutes before the announcement was made. I then got down to the platform whe two trains were waiting, both drivers acknowledged me, and said that they didn’t have a signal and that the would be a delay.

      I then got on the train, the place was swarming with viola and AT staff, who were doing their best, they told people that they were welcome to go and tag off and reminding them of the 20 minute rule. They also had staff in the info kiosks who appeared to be doing refunds. They also had staff by the gates telling people tagging on that if the trains didn’t start in 20 minutes they would need to tag off.

      Once the decision was made to suspend services, they advised that train tickets and hop cars were valid on all nz bus services, so I went upstairs and got a bus home.

      All in all I have to say Auckland transport did do a good job of communication in what was a trying situation.

  8. ‘from which the Auckland network is managed from’?

    The signalling problems are unfortunate, but without knowing the details it’s hard to know who or what exactly is to blame. Rob’s idea of an inquiry would figure things out; if nothing else, it seems that there aren’t adequate fail-safe measures in place. I don’t know if it’s necessary to have an independent inquiry after just two signalling failures that might have resulted from isolated fluke incidents, but if there are any more it’s something to consider.

  9. Struggling with the AT HOP system currently. Correct me if I am missing something but if a person gets on at a train station without the barrier systems and gets off further down the line I can see no incentive other than honesty to pay for the journey. If a ticket inspector is on board you can purchase one from him. Barriers are only in place for Britomart and Newmarket I think. Are there plans to extend for the whole network? Too many holes. AT HOP can not work in the current format unless there is NO option to buy tickets on the train, only fines.
    Can we expect the “journeys traveled” to drop (if the numbers are based on revenues) in the coming months as people dodge fares?

    1. – Apparently 1 in 3 trains arriving at a station will be checked by revenue protection officers (RPOs).

      – On-board sales will stops next year, officially they have already stopped.

      – Authorities are currently working to make it so the RPOs can issue fines, hopefully large ones.

      – Manukau will also be gated next year, and New Lynn is future proofed for gating but will not be. I know some people that read this blog would like to see many more gated.

      – I think the (hopefully short term) loss of fare revenue between non-gated stations will be over-ridden by the increase in paying customers from inner stations, where previously it was too crowded for the POs to move though the carriages. This will hopefully increase journeys travelled, although th number for the off-peak may decrease.

      – I agree that haste should be used in stopping on-board sales, introducing fines, and making be checked a common occurance.

  10. People calling for a “fail-safe” backup system are asking for the wrong thing. The system already is “fail safe”, in that if it fails then everything safely stops.
    What is needed is a “fail-proof” system, or better still, a “fool-proof” system that cannot be completely disabled by the occurence of a single failure or the actions of a fool, somewhere. How often do the Swiss Railways come grinding to a halt? Hardly ever. But the Swiss don’t have a penny-pinching, rail-averse National government skimping on everything except new motorways.

    1. Only time I ever experience a train in Switzerland that came to a halt was when someone pulled the emergency break, all the other times the announcements are conductors profusely apologising for the train being 1 minute late and hoping this won’t inconvenience the people on board.

    2. I don’t know how you would ‘fool proof’ a piece of fibre cable. At the end of the day, contractors the world over make mistakes and you could spend inordinate amounts of money trying to stop it happening when in fact the money could be put to better use building more infrastructure.

  11. In regards to most of the stations not having barriers, I was considering this in relation to the UK stations that I am familiar with.
    In London the only stations to have barriers are the ones that are manned, so there is always someone at the barriers in case there is a problem or someone gets stuck. So in the evenings for example all stations just open the barriers as I assume it is not economic to have someone standing there at 10pm at night, even if the station is busy like on a Friday night.
    So using the same logic in Auckland where most stations don’t have any staff – I would not expect any barriers.
    Does New Lynn have staff??

Leave a Reply

Your email address will not be published. Required fields are marked *