Safety should be the most important goal when designing and implementing new automation systems. Automation can play a critical role in enabling the human user to provide safe services. Automation used for other purposes (i.e., increased efficiency) should be designed and implemented only after considering the effects on human performance in the entire system through safety analyses that are not simple box checking exercises.
Automation solves many problems and can increase system safety, capacity and efficiency. However, new automation systems can also create new safety issues or worsen existing hazards. Such issues can include:
Automation designers should design and test systems rigorously, working with the future users from the initial planning stage, to determine the appropriate functions of the system, system displays / controls, normal operational procedures and procedures for handling failures and unexpected situations. Automation designers should develop and use an automation philosophy when making design choices. No new piece of automation should be introduced into live operations until it has been thoroughly tested in a variety of operational contexts to ensure that safety-critical functions perform in a way that supports the automation user in providing safe services.
Safety Management and Automation in Air Traffic Control: the future – Neil May
http://nats.aero/blog/2013/11/safety-management-automation-air-traffic-control-future/Humans and Automation: Use, Misuse, Disuse, and Abuse – Raja Parasuraman & Victor Riley
http://archlab.gmu.edu/people/rparasur/Documents/ParasRileyHF97.pdfThe Black Swan: The Impact of the Highly Improbable – Nassim Nicholas Taleb
http://www.nytimes.com/2007/04/22/books/chapters/0422-1st-tale.html?_r=0Quality Assurance in Safe Automation – Angela Summers & William Heam
http://sis-tech.com/wp-content/uploads/2011/05/Quality_Assurance_In_Safe_Automation.pdfNormal Accidents – Charles Perrow
http://esd.mit.edu/WPS/wplit-2003-01.pdfHumans: Still Vital After All These Years of Automation – Raja Parasuraman & Christopher D. Wickens
http://archlab.gmu.edu/people/rparasur/Documents/ParasuramanWickens08.pdf