An introduction to Safety Engineering

What is safety engineering?

Safety is about avoiding harm that results from accidents. In order to do that, we need to think about how accidents can be caused, and avoid the causes if possible, or mitigate them if we cannot remove them.

In safety engineering, we refer to things or conditions which have the potential to cause accidents as hazards. Hazards can be external to the system – such as an icy road, or a person in a duck suit crossing the road -, or they can be part of its design – such as petrol, electricity, heat or sharp edges.

Safety-critical systems

Self-driving cars are safety-critical systems. That is, they belong to a class of systems that have the potential to cause harm to people or the environment if they were to go wrong.

When we design and build systems like this, we need to ensure that safety – i.e. avoiding harm – is our priority. This means that we need to be able to assure ourselves (and anyone likely to be affected by the system) that we understand how the system will behave in all situations it is likely to encounter, and that we do everything we can to avoid accidents or reduce the severity of the consequences when we are unable to avoid them.

Demonstrating safety

Before the systems can be put into operation in the real world, engineers need to demonstrate their safety to regulatory bodies.

To do that, we need to have a clear understanding of potential dangers, and of the safety features we put in place to provide protection. These concepts are also useful in helping us persuade the public that the systems are safe enough to use. No system is ever absolutely safe (even a parked car has some risks associated with it): in safety engineering, we are working to provide systems that are ‘safe enough to accept for use’.

What are the hazards to avoid?

Some hazards are things that we can avoid: for example, we can change the design of a car so as to reduce the number of sharp edges, thus reducing the severity of the injuries that might be suffered if the car were to hit someone.

Other hazards are beyond our control: we cannot control the weather or the design of the road, for example. For those hazards, we need to include mitigations, to reduce the likelihood of accidents happening or to reduce their harmful effects if they do.

For example, we might build anti-skid systems, mandate specific kinds of tyres in winter, or make recommendations such as reducing speed in icy weather.

Safety requirements

The need to avoid or mitigate specific hazards is captured in safety requirements. In engineering, a requirement is an instruction to the designers or operators of a system specifying some behaviour or quality the system needs to have, or some condition that it must bring about or maintain.

For example, “The sub-system shall bring the vehicle to a halt within the prescribed stopping distance when commanded by the driver” is a requirement on a car’s braking sub-system. Safety requirements are those requirements that specify how a safety-critical system should handle hazards.

What are the main safety requirements?

There are several sources of safety requirements. Some come from the regulations which govern the development and operation of safety-critical systems: these generally concern well-known hazards within the domain (for example, for road vehicles, the environmental conditions in which vehicles are expected to operate in and things like maximum stopping distances are mandated by regulations and standards).

Others are specific to the individual system, and so need to be established during the development process. At each stage of the design – from the earliest conceptual phase through to the entry into service – hazard analysis is carried out.

How to identify hazards

A variety of techniques are used to identify potential hazards, and to assess their seriousness. Requirements are then developed, to specify ways in which these hazards should be addressed.

High-level safety requirements – about behaviours or properties of the system itself – are gradually made more precise as the system develops: they are translated into requirements on individual sub-systems or components which go to make up the system.

For example, a safety requirement on a car might state “the car will not skid on icy roads”. This will then be refined as the design develops, into more precise requirements about individual subsystems, for example, a requirement specifying the performance of an interlock in the braking system.

For a self-driving car, a high-level requirement that the AI system should be able to identify a human pedestrian with 100% accuracy in all lighting conditions will be translated into individual performance requirements on the various sensors, among other requirements.