What is Bash/Shell Scripting?

Before we look at scripting, let’s take a step back and break down what a shell is and how it relates to you and your computer.

What is a Shell?

We can crudely break a computer down into two main components: the operating system (e.g. UNIX) and the hardware (e.g. memory, graphics or CPU). Without hardware, there is very little that can be done with a computer. In order to communicate with this hardware, a piece of software, known as the operating system, is installed.

The operating system is typically broken down into two parts: the user space and the kernel. The kernel has direct access to the hardware and is responsible for sensitive tasks such as managing resources. Meanwhile, the user space acts as an interface, translating the needs of the user into the system calls that get run by the kernel.

So, now we get to the original question, “what is a shell”? A good analogy here is a hazelnut. At the core of a hazelnut is a soft, edible kernel, surrounded by the hardened shell to protect it. In computing, the shell is the generic name for an application or program which allows access to the system’s resources. You may also see this referred to as the command line interpreter or CLI. It acts as a wrapper, protecting the sensitive kernel inside.

What is Bash?

There are many types of shell available, each of which has a distinctive syntax. The first, the Thompson shell, was released in 1971 by Ken Thompson. However, this had many limitations leading Stephen Bourne to release his Bourne shell six years later, in 1977. Providing a more functional interaction with the operating system, the Bourne shell not only served as a CLI, but introduced variables, loops and control flows into shell scripts – useful concepts we’ll be introducing in more detail shortly!

Unfortunately, the most widely used shells at that time were not freely available. In order to bridge this divide, Brian Fox developed the Bourne-Again Shell (Bash) in 1988 for the GNU project. Bash is now the default shell on most Linux systems. Thus, understanding how to use it is a key part of any data scientist’s toolbox.

Depending on your operating system, your default shell may not be Bash. To check which shell you are using, you can run the following command:

echo $SHELL

This will return the full path of your interpreter. If your interpreter is Bash you will see a path similar to:

/bin/bash

Alternatively, you may also see the Z, C or TC shells which would have paths similar to one of the following:

/bin/zsh

/bin/csh

/bin/tcsh

You can get a full list if the shells that are available on your system by running:

cat /etc/shells

To change the shell your computer uses temporarily (i.e. in the current terminal) you can type the interpreter executable name. For example, to switch your shell to csh, you can enter:

csh

This is a good way to play with other shells without making a permanent change. To switch back to your existing shell, press Ctrl-D.

If you like a particular shell and want to make the change permanent (i.e. set in all future terminals by default), you can use the chsh command with the -s option (bypassing the editor) and the path to the interpreter you want to use.

chsh -s /bin/bash

Note: you may need to log out (close the terminal) and log back in again (open a new terminal) for this change to take effect.

What is Shell Scripting?

Let’s say you want to know what proportion of lines in a file contain a particular phrase. First, you would need to determine the total number of lines in that file. Then, count the number of lines containing the phrase. Finally, you would need to perform a calculation to get the proportion of lines with that phrase. Simple, right?

Now let’s say you need to do this for 100 files, once for each file and record the results…not so simple, eh?

You could take the extremely tedious approach of manually running the commands 100 times. But, how long would it be before you made a mistake? How would you track the output of all your commands? What if you needed to run the same thing again tomorrow?

Bash/Shell Scripting

No matter how careful you are, manual errors always creep in during these kinds of repetitive tasks. We have a simple path to salvation from this tedium…Bash/shell scripting!

Bash scripting is a powerful way to automate tasks which you need to execute on a computer. Put simply, a Bash script is just a collection of Bash commands which are kept in a text file.