Rogue code led to Gmail shutdown

A rogue piece of software, which triggered a cascading failure through Google’s data centres around the world, led to the shutdown of its Gmail system last week.The failure, during routine maintenance at one of the company’s European data centres, was an unforeseen side-effect of a software program that was written in-house, said Nelson Mattos, vice-president of engineering.”We’re not perfect, we make mistakes,” he said.But experts said that anticipating how software would behave in circumstances such as these was something that even corporate IT departments with far less engineering sophistication than Google were expected to master.
http://www.ft.com/cms/s/0/c5dd4574-06a3-11de-ab0f-000077b07658.html