Automation systems must provide evidence that they support safe human performance in terms of workload, situation awareness, user confidence, and decision making.

Safety should be the most important goal when designing and implementing new automation systems. Automation can play a critical role in enabling the human user to provide safe services. Automation used for other purposes (i.e., increased efficiency) should be designed and implemented only after considering the effects on human performance in the entire system through safety analyses that are not simple box checking exercises.

Why It's Important

Automation solves many problems and can increase system safety, capacity and efficiency. However, new automation systems can also create new safety issues or worsen existing hazards. Such issues can include:

  • increasing system complexity and tight coupling;
  • overreliance on automation by users;
  • under-use of automation by users;
  • loss of manual backup skills of users;
  • inappropriate use of the system by users; and
  • inappropriate automation of tasks by system designers.

Design and Implementation Considerations

Automation designers should design and test systems rigorously, working with the future users from the initial planning stage, to determine the appropriate functions of the system, system displays / controls, normal operational procedures and procedures for handling failures and unexpected situations. Automation designers should develop and use an automation philosophy when making design choices. No new piece of automation should be introduced into live operations until it has been thoroughly tested in a variety of operational contexts to ensure that safety-critical functions perform in a way that supports the automation user in providing safe services.




Further Reading on Safety Assurance

Safety Management and Automation in Air Traffic Control: the future – Neil May
http://nats.aero/blog/2013/11/safety-management-automation-air-traffic-control-future/

Humans and Automation: Use, Misuse, Disuse, and Abuse – Raja Parasuraman & Victor Riley
http://archlab.gmu.edu/people/rparasur/Documents/ParasRileyHF97.pdf

The Black Swan: The Impact of the Highly Improbable – Nassim Nicholas Taleb
http://www.nytimes.com/2007/04/22/books/chapters/0422-1st-tale.html?_r=0

Quality Assurance in Safe Automation – Angela Summers & William Heam
http://sis-tech.com/wp-content/uploads/2011/05/Quality_Assurance_In_Safe_Automation.pdf

Normal Accidents – Charles Perrow
http://esd.mit.edu/WPS/wplit-2003-01.pdf

Humans: Still Vital After All These Years of Automation – Raja Parasuraman & Christopher D. Wickens
http://archlab.gmu.edu/people/rparasur/Documents/ParasuramanWickens08.pdf