How to write robust bash scripts

Sometimes, despite having the very best intentions, subtle issues can creep into your script causing it to fail with unintended consequences. Fortunately, there are commands available to help with minimising these issues. One of these is the set command.

The set command

Let’s take a look at how the set command can help us write robust and secure Bash scripts.

First, how does the set command work? Using the set command allows us to customise the environment in which our scripts are run.

The general syntax for the set command is:

set [options]

There are more than a dozen options available for the set command. To view them, you can run the following command:

help set

In this article, we’ll be focusing on the most commonly used options.

Using set -e to catch errors

Sometimes, commands within your script may fail but, the downstream commands will continue to run. This can be extremely frustrating if you don’t see the error and assume that, as the script is completed, everything has worked as expected.

Here’s an example. First, we will try to change it into a directory called foo and then list the contents of that foo directory. The key here is that the foo directory doesn’t actually exist so, we can’t get its contents.

#!/usr/bin/env bash
 
cd foo
ls

What happens when we run our script?

script.sh: line 3: cd: foo: No such file or directory
File1 File2

Notice that our script generated an error when the system couldn’t find our foo directory. But, because there wasn’t an exit code, the remaining commands in the script also-ran.

Unfortunately, this listed the contents of our current working directory and not the foo directory as intended. Imagine if this was part of a long series of output commands and we missed the error….we may accidentally assume that our script ran correctly!

Fortunately, the set -e command comes to our rescue by ensuring that the script will fail whenever an error occurs, no matter the exit code. Try adding set -e to the top of your script:

#!/usr/bin/env bash
 
set -e
 
cd foo
ls

Bingo! This time, we can see that the script terminates as soon as it reaches the first error.

script.sh: line 5: cd:foo: No such file or directory

Using set -u to catch variables that don’t exist

By default, when executing a script, Bash will just ignore variables that don’t exist. In most cases, you won’t want this behaviour as it can have unexpected consequences!

In this example, we will first try to output a variable, $foo, which doesn’t exist and then try to output a simple string, bar.

#!/usr/bin/env bash
 
echo $foo
echo bar

When we run this script, we get the following output:

bar

Notice that the system outputs a blank line for echo $foo. This is because Bash is ignoring $foo as it doesn’t exist.

If we want the script to exit with an error instead of continuing on silently, we can add the set -u command at the top of our script.

#!/usr/bin/env bash
 
set -u
 
echo $foo
echo bar

This will result in our script exiting with the following error:

script.sh: line 6:foo: unbound variable

Notice, our script terminates before running the second echo command.

Displaying executed commands while the script is running with set -x

Another default Bash behaviour is to only display results once a script has finished. This can be especially frustrating when you need to debug scripts that take a long time to run.

Let’s take an example script that outputs two simple strings, foo and bar.

#!/usr/bin/env bash
 
echo foo
echo bar

The output from this script would be:

foo
bar

Now, what if we want to know which command is producing each of the results? To find this out, we can use the set -x command which outputs the executed command before printing the command result.

#!/usr/bin/env bash
 
set -x
 
echo foo
echo bar

Running this script would give the following output:

+ echo foo
foo
+ echo bar
bar

As you can see, before executing each of the echo commands, the script first prints the command to the terminal, using a + to indicate that the output is a command. This can be especially handy when you want to debug long scripts.

Combining set options in a single command

Most of the time, you will want to use all of these options together. Instead of writing the commands out, one command per line, we can combine the options into a single command:

set -eux

Using the set command is essential to building robust Bash scripts. Not only is it part of good scripting practices but, will also save you a lot of time and frustration!

If you’d like to learn more about bash scripting, check out the course, from the Wellcome Connecting Science online course, below.