Audio in the Game

As mentioned earlier, vision is the primary channel through which we acquire most of our information, yet sound in games also conveys information and possesses artistic merit, much like visuals do.

In the context of gaming, audio is not merely an accompaniment; it is an integral part of the experience that can significantly enhance immersion and engagement. Sound effects, music, and voice acting work together to create a rich auditory landscape that complements the visual elements, contributing to the overall atmosphere and storytelling.

Background music (BGM), or the score, is designed to set the mood and tone of the game. It can evoke emotions and heighten tension during key moments, such as battles or dramatic scenes. For instance, The Legend of Zelda series uses orchestral music to create an epic atmosphere that matches the heroic journey of Link. The music changes based on the environment and situation, helping to guide the player through different moods and feelings. As you know, BGM differs from other musical forms such as film scores, concert music, and pop songs in several significant ways, tailored specifically to the interactive medium of video games. Because BGM must adapt to the dynamic nature of gameplay, responding to player actions, game events, and the state of the game world. This can involve changes in tempo, intensity, or even entire musical themes based on whether the player is exploring, battling enemies, or solving puzzles. At the same time, often structured in loops or modules, BGM is designed for endless replay without becoming monotonous. Composers need to craft music that maintains interest over repeated listening while seamlessly looping. Unlike film or theater music where timing is fixed, game music must accommodate the variable pacing controlled by the player. It needs to be scalable, extended, or shortened as needed to match the player’s progress through the game. In terms of BGM creation, because it is designed to enhance the atmosphere of the game world and help players immerse themselves in the game environment, it usually requires a clear theme to provide an emotional backdrop for different scenes and moods, such as peaceful villages, tense battles, mysterious dungeons, etc., guiding players’ emotions. BGM sometimes also includes lyrics, but unlike pop music or songs, the lyrics in games tend to focus more on the connection with the game’s theme rather than independent narratives. Theme songs or songs related to certain plots may contain specific professions, characters, or game features.

Voice acting brings characters to life, adding depth and personality to their dialogue. A well-executed script, combined with talented voice actors, can make the narrative more compelling and the characters more relatable. It can also serve to localize the game, making it accessible to a global audience by adapting cultural nuances and language, and voice actors for game characters and the games themselves mutually reinforce each other, leading to increased recognition and improved opportunities for development. Video game voice acting must be versatile and adaptable to the interactive nature of games. Actors often record lines under varying conditions, including reacting to events that the player can trigger or alter. Interestingly, if voice actors are unfamiliar with the game and fail to imagine the content they are dubbing, especially for shorter interjections (often accompanying attacks) or reactions (typically accompanying being hit), the resulting voices may struggle to blend into the game. A relevant example can be found in the English dubbing of Neuvillette in Genshin Impact. No offense intended, but the attack voices make the character appear overly forceful, which is difficult to coordinate with the character’s animation and conveys effortlessness.

Firstly, SFX must react to player actions and game events in real time. Unlike pre-recorded soundtracks in films, which are synced to a predetermined timeline, game SFX is dynamically triggered by player movements, interactions, or changes in the game environment. These effects are often layered to construct a rich auditory landscape. This includes ambient noises, specific action sounds, and responsive sounds that result from player activities. This necessitates that sound designers compile a comprehensive library of sounds that can be activated under various circumstances, ensuring that the audio remains fresh and relevant, irrespective of the player’s chosen path through the game.

The artistry of sound design in games is often overlooked, yet it plays a critical role in shaping the player’s experience. In its essence, gaming revolves around the creation of a virtual realm, and consistent with the classical magic circle theory, games erect a world that is segregated from the tangible world with respect to both time and space. From a cognitive viewpoint, gamers employ digital signals to conceptualize, rationalize, and commit to a simulated universe, and there is no fundamental disparity in the constitution of this universe whether it is assembled through visuals or sounds. Accordingly, beyond our typical engagement with games through visual channels, adopting auditory information as the key pathway for acquiring and processing game-related data is perfectly plausible. Contemporary spatial audio technologies offer the potential for a richer construction of 3D spaces, providing enhanced cognitive experiences.

In a past academic project, my students embarked on an ambitious endeavor to develop a game that relied solely on sound, eschewing any form of visual presentation on the screen. Participants were tasked with moving through the game environment by pressing keys in accordance with the auditory information they received, using the disparity in sound between their left and right ears to orient themselves and respond to occurrences, with the ultimate goal of escaping from a locked room. When we conducted peer reviews and shared experiences during class, this particular game garnered substantial praise and insightful recommendations. It offered a unique insight into the world of individuals with visual disabilities, enabling us to understand and simulate the difficulties they face when relying exclusively on auditory cues to navigate their surroundings. Beyond presenting a novel and challenging gameplay mechanic, it served as a powerful tool for fostering empathy and broadened our perspectives on sensory experiences that differ from our own.

The gaming industry is gradually recognizing the importance of accessible design, ensuring that all players, including those with visual impairments, can enjoy electronic games. Games typically employ methods such as audio descriptions and narration, text-to-speech, haptic feedback, and sound effect cues to assist players with visual impairments. But experienced designers who have truly delved into accessible design will tell you that designing barrier-free games for visually impaired players goes beyond simply providing voice instructions or translating visual content. It requires a shift from visual thinking to auditory thinking, where the very foundation of game design considers how levels and information can be conveyed to players through auditory cues from the outset. This is a task that, even when you understand the principles, is hard to get right on the first try. I recommend watching the work logs of the indie game Red&Black for insights; you can find them at the following URL: https://www.bilibili.com/video/BV18M4y1b7TL/