DALLAS – Southwest Airlines struggled to recover Thursday from a massive computer outage, as several hundred more flights were cancelled or delayed around the country for a second straight day.
Airline executives said a router breakdown set off a chain of failures in critical technology systems a day earlier. Backup systems didn’t work as expected, they said.
Southwest said that “most systems” had recovered and were functioning by Thursday morning. After “a day of recovery,” the airline hoped to be operating normally on Friday, CEO Gary Kelly said.
By Thursday evening, Southwest had cancelled more than 600 flights and 2,200 more were delayed, according to tracking service FlightStats Inc. Those numbers were very similar to Wednesday, when the airline said it cancelled nearly 700 flights and FlightStats said 2,100 were delayed.
For hours after the outage started, airline employees had to check in passengers manually and couldn’t take new reservations. The company’s website crashed — Kelly estimated that Southwest might have lost between $5 million and $10 million in ticket sales because customers couldn’t book flights.
“We have significant redundancies built into our mission-critical systems, and those redundancies did not work,” Kelly told reporters on a conference call. “We need to understand why, and make sure that that doesn’t happen again.”
Southwest has an aging technology infrastructure, but Kelly said the airline has been making “significant investments” to upgrade it. It expects to replace the longstanding reservations system next year — at a cumulative cost of $500 million — and replace other key systems over the next three to five years.
Chief Operating Officer Mike Van de Ven said when a router failed Wednesday, it slowed the airline’s technology systems so much that other functions weren’t usable. He said router failures aren’t uncommon, but this outage was unusually severe. It took crews about 12 hours to restore most systems to working order.
When the outage hit, Southwest took the unusual step of briefly holding all departing flights at their gates.
With so many flights cancelled on Wednesday, Southwest knew that a large number of passengers would try to fly Thursday. To bounce back better, the airline reduced its load, cancelling more than 200 flights or 5 per cent of its daily schedule even before the day’s first flight. It warned passengers to expect long lines, and to arrive at the airport early.
Southwest suffered a similar outage last October, resulting in about 800 delayed flights. It blamed that breakdown on a faulty software application.
David Koenig can be reached at http://twitter.com/airlinewriter