From Whispers to Snowstorms: Explore the Immersive World of Spatial Sound Design in Unreal Engine 4
Back in 2018, I delivered audio for a Virtual Reality project for a major London-based entertainment company using Unreal Engine 4 (UE4).
It was my second time working with UE4, and the entire process was smooth and straightforward—much more comfortable and with better results than Unity. This experience inspired me to dive deeper into the UE4 platform and create my own Spatial Audio Project.
Sounds simple, right?
I have a collection of songs I created using Artificial Intelligence, which I thought would be easy to deploy into UE4 and bring to life…
Take a listen to them:
Well, it turns out things aren’t that easy.
How can a Sound Designer build an entire world inside Unreal?
The answer is research. There are countless tutorials online. A basic knowledge of C++ and game engine mechanics helps—a lot.
Even without any previous knowledge of building materials, meshes, colliders, and so on?
Yes. Like Unity, there are plenty of free packages of high-quality materials, particles, and meshes to download and start working with.
How do I make a VR player interact with the environment I intend to create? How do things work (apart from Audio) inside UE4?
That’s the biggest question. Fortunately, there’s an initial template available that provided me with the necessary abilities for interaction and movement.
This is where I started with the basic template:
And this is what I’ve created in the end:
Once things were close to how I envisioned them, I added more interactivity and created new ways of experiencing music in a VR environment.
Now I can confidently say that I’m able to deliver at least a reasonable prototype in Virtual Reality and First-Person modes in Unreal. This includes not just the audio, but a basic package of interactions, environments, designs, functionalities, and more.
The project
- Note: All videos were recorded while I was using my Head-Mounted Display (HMD), so only the right eye view is shown on screen.
To download the full experience for HTC Vive, go to this link – https://1drv.ms/u/s!Ass2QbRc4bnYhV-4ANOGweLNdZwd?e=gNGNpt
Each area of the project offers a unique way to experience audio.
To start, you’ll hear a voice announcing that the experience is a “Journey into Sound”.
This is the famous speech by Geoffrey Sumner from the “Train Sequence” on the 1958 album “A Journey Into Stereo Sound”, featuring sound effects, music, and noises from London FFS Recordings.
As you begin exploring the space, you’ll encounter various possibilities of Spatial Sound, such as this blue cloud with whispers.
There’s also a song where the central big red dot provides a sweet spot perspective, but you can move behind each particle emitter to listen to individual parts separately.
You can then interact with objects in the scene. The lamp, pipes, big black machine in the corner, statue, and even the wires all have their unique sounds.
This girl, courtesy of the fantastic 3D artist Miguelangelo Rosario, originally had no sound. So I’ve given her a complaint: “What are you looking at?”
🙂
There’s even a Binaural Sound emitter. Stand in the middle of the field to hear different frequencies in each ear, creating a binaural effect for concentration (16Hz directly to your brain). A future idea: create a Binaural Beats relaxation environment in VR.
Another song follows the same concept as the first. Move around to hear each stem separately, or stay in the middle for the sweet spot.
The dancing girls, also courtesy of Miguelangelo, move to the tune from a large boom box. Stand behind one of them to hear how she occludes the original audio while dancing.
Step outside to experience a snowstorm.
Visit the graveyard to hear fire torches, eerie music near the tombs, and a crow in a nearby tree.
In another direction, you’ll find a greenhouse filled with birdsong and a Bossa Nova tune. You’ll also find more information about the gigantic statue there.
At the end of the footbridge, you have three options:
- Directly in front of you, play with spheres and their sounds. Just grab one and throw it.
- To your left, enter a dark room with more whispers, another eerie tune, and a candle in the center.
- If you venture further, you’ll meet the Groove Master (another of Miguelangelo‘s animations). This dancing guy is enjoying himself while listening to another song.
The project showcases excellent examples of Spatial Audio, occlusions, loops, placements, and the audio possibilities within Unreal Engine.
If you own an HTC Vive connected to a Windows computer, you can download and experience the project yourself.
Here’s the link – https://1drv.ms/u/s!Ass2QbRc4bnYhV-4ANOGweLNdZwd?e=V7Hwt6
For a complete walkthrough, check out this video:
I’ve also included some additional images from the development process.
Have fun!
Cheers,
Billy.
Deixe um comentário