Want to keep learning?

This content is taken from the Monash University's online course, Creative Coding. Join the course to learn more.
An image from the w4_04 sketch that links sound and image
The w4_04 sketch

Using sound in Processing

Sound is an important part of our environment and our lives. Yet the value of sonic spaces and acoustic environments are often treated as secondary or ignored in our human-made world. We recognise the presence of unwanted sound as ‘noise pollution’, yet the design of our cities, buildings and public spaces often neglect the idea that the built environment is a sonic experience as well as a visual and tactile one.

So far all of our sketches has been silent, which may not be a bad thing in today’s noise-filled world. But sound can also play an important role in generative and code-based art.

Conventionally, sound is often used in a way that is subservient to image, or used to simply to reinforce the emotions or actions of images. In contrast, sound and sound art ask the listener to actively engage in the act of listening as opposed to just hearing.

Open the w4_4 sketch and run it. This sketch creates a generative visual pattern synchronised with the playback of pre-recorded sounds.

To play back audio, Processing needs to use one the Sound library. We installed this in the Installing Processing: Windows and the Installing Processing: Mac OS X steps in Week 1. You will need to install the sound library for this sketch to run.

Notice that we import the functions provided by this library in line 16:

// setup the sound library and audio player
import processing.sound.*;     // import the sound libraries into Processing

The import command reads in the necessary definitions of type, variables, objects and so on, so that they can be used in the sketch. We declare a variable (an array of objects), soundFile of type SoundFile (yes, UPPER or lower case is important) to represent each of the .wav files that are pre-recorded sounds .

As usual, the code in setup() does the initialisation:

// load samples from the "Data" directory
soundFiles = new SoundFile[3];  // 3 SoundFiles - 1 for each sound
soundFiles[0] = new SoundFile(this, "sound01.wav");
soundFiles[1] = new SoundFile(this, "sound02.wav");
soundFiles[2] = new SoundFile(this, "sound03.wav");

Each soundFiles array element is an object so we need to create a new object of type SoundFile and initialise it with the name of the sound file the object represents. In object-oriented terminology, SoundFile is called a class. this is a special variable that represents the Processing object within which this code resides – we’re getting out of our depth here in object-oriented programming, so don’t worry too much about the details of what all this means for now (we’ll cover objects and classes next week). The string passed (e.g. "sound01.wav") is the name of the file that stores the sound we want to play back in our sketch. As with the image-reading code, sounds reside in the ‘data’ directory inside the folder of the Processing sketch itself.

The basic thing to understand here is that we’re creating a new ‘thing’ in memory and that thing is of type SoundFile. We’re making 3 of them because we want to use 3 different sounds and storing each object in an array.

So, we have loaded three different sounds that are ready to be played on our command.

 // randomly select a sound and play it
 selectSound = (int) random(3);

This code just selects a random number between 0 and 2 and then plays the sound indexed by this number.

If we want to play the same sound again, we need to “rewind” the file cue point back to the beginning using the cue() member function:

  soundFiles[selectSound].cue(0); // rewind the cue point to the begining of the sound

Learn more about the Processing Sound Library

Go to the See also section for a link to the documentation for the Processing sound library and the Sound foldout in the next step.

Share this article:

This article is from the free online course:

Creative Coding

Monash University

Get a taste of this course

Find out what this course is like by previewing some of the course steps before you join: