Skip main navigation

How to write robust bash scripts

This article will walk you through how to write robust bash scripts that will avoid any issues creeping in.
© Wellcome Genome Campus Advanced Courses and Scientific Conferences

Sometimes, despite having the very best intentions, subtle issues can creep into your script causing it to fail with unintended consequences. Fortunately, there are commands available to help with minimising these issues. One of these is the set command.

The set command

Let’s take a look at how the set command can help us write robust and secure Bash scripts.

First, how does the set command work? Using the set command allows us to customise the environment in which our scripts are run.

The general syntax for the set command is:

set [options]

 

There are more than a dozen options available for the set command. To view them, you can run the following command:

 

help set

 

In this article, we’ll be focusing on the most commonly used options.

 

Using set -e to catch errors

 

Sometimes, commands within your script may fail but, the downstream commands will continue to run. This can be extremely frustrating if you don’t see the error and assume that, as the script is completed, everything has worked as expected.

 

Here’s an example. First, we will try to change it into a directory called foo and then list the contents of that foo directory. The key here is that the foo directory doesn’t actually exist so, we can’t get its contents.

 

#!/usr/bin/env bash
 
cd foo
ls

 

What happens when we run our script?

 

script.sh: line 3: cd: foo: No such file or directory
File1 File2

 

Notice that our script generated an error when the system couldn’t find our foo directory. But, because there wasn’t an exit code, the remaining commands in the script also-ran.

Unfortunately, this listed the contents of our current working directory and not the foo directory as intended. Imagine if this was part of a long series of output commands and we missed the error….we may accidentally assume that our script ran correctly!

Fortunately, the set -e command comes to our rescue by ensuring that the script will fail whenever an error occurs, no matter the exit code. Try adding set -e to the top of your script:

 

#!/usr/bin/env bash
 
set -e
 
cd foo
ls

 

Bingo! This time, we can see that the script terminates as soon as it reaches the first error.

 

script.sh: line 5: cd:foo: No such file or directory

 

Using set -u to catch variables that don’t exist

 

By default, when executing a script, Bash will just ignore variables that don’t exist. In most cases, you won’t want this behaviour as it can have unexpected consequences!

 

In this example, we will first try to output a variable, $foo, which doesn’t exist and then try to output a simple string, bar.

 

#!/usr/bin/env bash
 
echo $foo
echo bar

 

When we run this script, we get the following output:

 

bar

 

Notice that the system outputs a blank line for echo $foo. This is because Bash is ignoring $foo as it doesn’t exist.

 

If we want the script to exit with an error instead of continuing on silently, we can add the set -u command at the top of our script.

 

#!/usr/bin/env bash
 
set -u
 
echo $foo
echo bar

 

This will result in our script exiting with the following error:

 

script.sh: line 6:foo: unbound variable

 

Notice, our script terminates before running the second echo command.

 

Displaying executed commands while the script is running with set -x

 

Another default Bash behaviour is to only display results once a script has finished. This can be especially frustrating when you need to debug scripts that take a long time to run.

 

Let’s take an example script that outputs two simple strings, foo and bar.

 

#!/usr/bin/env bash
 
echo foo
echo bar

 

The output from this script would be:

 

foo
bar

 

Now, what if we want to know which command is producing each of the results? To find this out, we can use the set -x command which outputs the executed command before printing the command result.

 

#!/usr/bin/env bash
 
set -x
 
echo foo
echo bar

 

Running this script would give the following output:

 

+ echo foo
foo
+ echo bar
bar

 

As you can see, before executing each of the echo commands, the script first prints the command to the terminal, using a + to indicate that the output is a command. This can be especially handy when you want to debug long scripts.

 

Combining set options in a single command

 

Most of the time, you will want to use all of these options together. Instead of writing the commands out, one command per line, we can combine the options into a single command:

 

set -eux

Using the set command is essential to building robust Bash scripts. Not only is it part of good scripting practices but, will also save you a lot of time and frustration!

If you’d like to learn more about bash scripting, check out the course, from the Wellcome Connecting Science online course, below.

© Wellcome Genome Campus Advanced Courses and Scientific Conferences
This article is from the free online

Bioinformatics for Biologists: An Introduction to Linux, Bash Scripting, and R

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education