PS5 want to immerse you in 3D sound, how does that work?

New game consoles are often a vehicle for new technology, and the PlayStation 5 is no different. For example, Sony is introducing Tempest 3D AudioTech for the console, a technology that must place sounds very accurately and in 3D around the player.

The first game to come out with the technology is ‘Returnal’, a horror game set in space, on a planet that is hostile to the main character to say the least. Horror is traditionally a genre that can do a lot with sound (there are levels from the ancient ‘Thief’ games whose soundtrack still inspires nightmares), so the choice for ‘Returnal’ as the first taste is not out of the blue. We tested the game with the accompanying headphones and can report from experience that it is indeed scary when you hear a monster literally breathing behind you.

Microseconds difference

But how does that work? “There are mathematical and physical rules governing how our brain interprets sound that enters your ears,” says Loic Couthier, sound designer at Sony. ‘For example, if sound comes from the right, it will reach your right ear faster than your left ear. That sound in the left ear will also be muffled a bit because your body is still in between. The shape of your ear also has an influence here. It’s about sometimes very small differences in signals coming from your ears, which allow your brain to analyze where the sound is coming from, and we’ve analyzed that.’

Once you have those parameters, you can apply them in a game world. “For example, we decide that a sound from behind your character’s left will come into the game, and then we’ll take the steps to get your brain to interpret it that way,” Couthier continues. ‘It also only works with headphones because, for example, you also hear a right speaker with your two ears, so we can’t inject the sound into one ear.’

Much of the system revolves around the way you are used to receiving sound. The PS5 gives you a number of settings with which you can, for example, indicate the size of your head, a matter of getting as close as possible to the shape of your own body. “It’s about microseconds and nanoseconds, very small differences between when the sound reaches one ear and the other,” Couthier says, and that’s a little different for everyone.

Returnal is the first game to work with 3D sound technology. © Sony

Surround vs 3D

However, the idea of ​​sound coming from different directions has been around for some time. The concept of ‘surround sound’ has been common for some time, especially in (home) cinema. Systems like Dolby Atmos and Auro 3D try to put the sound in your living room in such a way that it feels like it’s all around you.

‘Surround sound is limited to two axes’ explains Couthier when we point this out to him. ‘In 3D audio, sound is measured on a sphere, so that you have all dimensions. With surround sound, anything that happens below you still has to sit on those two axes. With 3D you can also put it under your feet.’

VR

The fact that an idea that has been around for some time is now getting this elaboration has a lot to do with virtual reality, the technology that pre-eminently wants to give gamers the experience of an immersive digital world. “We started with PSVR,” explains Couthier. “That was the first version of this, and it also made sense to have 3D sound in virtual reality. With the PS5, we’ve now made the decision to support that audio more broadly across the platform.”

In addition, a lot has to do with the computing power of the console. “We can now run this at an affordable price in terms of hardware,” he says. ‘Because 3D sound requires a lot of power, it is four times heavier to run a 3D mix than, for example, a more classic 7-1 mix. It adds an extra dimension.’

Evolution of sound

3D is fun, but often remains a gimmick. ‘I do think it will be used more widely,’ says Couthier. ‘It’s about the evolution of sound. We are now going from 2D to 3D. In ‘Returnal’ it makes for very cool moments, but the best thing about this technology is that it allows you to hear the surroundings more precisely. For example, in a game you could avoid enemies before you see them. That is a sensation that works well in shooting games, for example. In addition, the experience, the feeling that you are inside that world, is workable for any kind of 3D game.’

The intention is that the technology is not necessarily limited to one console. That seems especially useful for games that are launched on multiple platforms. “Any platform can support 3D audio,” says Couthier. A game then has to place the sounds in the right place when programming, and different game consoles should be able to place them in the right place in the virtual sphere on their own platform. ‘In cinema you have a Dolby Atmos mix that can be played on any system, be it in surround sound or on a mono TV. A game like ‘Returnal’ has the same thing. That mix exists in 3D and the PS5 decodes that as 3D Audio; If you bring that game to another platform, they can have their own decoders.’

The first game to come out with the technology is ‘Returnal’, a horror game set in space, on a planet that is hostile to the main character to say the least. Horror is traditionally a genre that can do a lot with sound (there are levels from the ancient ‘Thief’ games whose soundtrack still inspires nightmares), so the choice for ‘Returnal’ as the first taste is not out of the blue. We tested the game with the accompanying headphones and can report from experience that it is indeed scary when you hear a monster literally breathing behind you. But how does that work? “There are mathematical and physical rules governing how our brain interprets sound that enters your ears,” said Loic Couthier, Sony sound designer. ‘For example, if sound comes from the right, it will reach your right ear faster than your left ear. That sound in the left ear will also be muffled a bit because your body is still in between. The shape of your ear also has an influence here. It’s about sometimes very small differences in signals coming from your ears, which allows your brain to analyze where the sound is coming from, and we’ve analyzed those. ‘Once you have those parameters, you can start applying them in a game world. “For example, we decide that a sound from behind your character’s left will come into the game, and then we’ll take the steps to get your brain to interpret it that way,” Couthier continues. ‘It also only works with headphones because, for example, you also hear a right speaker with your two ears, so we can’t inject the sound into one ear.’A lot of the system revolves around the way you are used to receiving sound. . The PS5 gives you a number of settings with which you can, for example, indicate the size of your head, a matter of getting as close as possible to the shape of your own body. “It’s about microseconds and nanoseconds, very small differences between when the sound reaches one ear and the other,” Couthier says, and it’s a little different for everyone. The idea of ​​sound coming from different directions exists. however, for a long time. The concept of ‘surround sound’ has been common for some time, especially in (home) cinema. Systems like Dolby Atmos and Auro 3D try to put the sound in your living room in such a way that it feels like it’s all around you. ‘Surround sound is limited to two axes’ explains Couthier when we point this out to him. ‘In 3D audio, sound is measured on a sphere, so that you have all dimensions. With surround sound, anything that happens below you still has to sit on those two axes. You can also put it under your feet with 3D.’The fact that an idea that has been around for some time is now being developed has a lot to do with virtual reality, the technology that is pre-eminently intended to give gamers the experience of an immersive digital world. “We started with PSVR,” explains Couthier. “That was the first version of this, and it also made sense to have 3D sound in virtual reality. With the PS5, we’ve now made the decision to support that audio more broadly across the platform.” A lot also has to do with the processing power of the console. “We can now run this at an affordable price in terms of hardware,” he says. ‘Because 3D sound requires a lot of power, it is four times heavier to run a 3D mix than, for example, a more classic 7-1 mix. It adds an extra dimension.’3D is fun, but often remains a gimmick. ‘I do think it will be used more widely,’ says Couthier. ‘It’s about the evolution of sound. We are now going from 2D to 3D. In ‘Returnal’ it makes for very cool moments, but the best thing about this technology is that it allows you to hear the surroundings more precisely. For example, in a game you could avoid enemies before you see them. That is a sensation that works well in shooting games, for example. In addition, the experience, the feeling that you are inside that world, is workable for any kind of 3D game.’ The intention is that the technology is not necessarily limited to one console. That seems especially useful for games that are launched on multiple platforms. “Any platform can support 3D audio,” says Couthier. A game then has to place the sounds in the right place when programming, and different game consoles should be able to place them in the right place in the virtual sphere on their own platform. ‘In cinema you have a Dolby Atmos mix that can be played on any system, be it in surround sound or on a mono TV. A game like ‘Returnal’ has the same thing. That mix exists in 3D and the PS5 decodes that as 3D Audio; If you bring that game to another platform, they can have their own decoders.’

We would love to give thanks to the writer of this short article for this remarkable web content

PS5 want to immerse you in 3D sound, how does that work?

Bofads