PlayStation.Blog

The Joshua Bell VR Experience Out Today on PS VR

13 1
The Joshua Bell VR Experience Out Today on PS VR
1 Author Reply

Today, we are releasing an exciting new musical experience for PlayStation VR. In partnership with award-winning violinist and Sony artist Joshua Bell, we’ve brought to life a live studio performance of Brahms’ Hungarian Dance No. 1 in full 360-degree VR.

With adaptive audio and positional tracking, the Joshua Bell VR Experience will transport you into the session and allow you to immerse yourself completely in the performance from every direction. The Experience is available now at PlayStation Store as a free download for PlayStation VR.

Joshua Bell is a celebrated, Grammy award-winning violinist with a career spanning more than 30 years as a soloist, chamber musician, recording artist, and conductor. Generally regarded as one of the finest classical musicians of this generation, he just completed a critically-acclaimed residency at the Kennedy Center for the Performance Arts and National Symphony Orchestra in Washington D.C. He is also a technology enthusiast.

When he tried an early PlayStation VR prototype a few years back, he was overwhelmed by its potential. He wondered how it could be used for classical music. At the same time, our group here in the UK happened to be researching new techniques for filming live events, particularly with music, and it seemed like the perfect opportunity to work together and collaborate with some of our other Sony companies.

We filmed at Air Studios’ Lyndhurst hall in London. Not only is it one of the best rooms in the world for recording orchestral instruments, but it also provides a stunning background for the performance. Lyndhurst Hall has been used to record the soundtracks to many famous movies and it is often used for our PlayStation games, too.

Using a number of Sony FDR-X1000V cameras, we filmed a high quality 360 degree video of Joshua’s brilliant performance with his accompanist Sam Haywood. Now with most 360 videos, you can only watch from a fixed point in space, but PlayStation VR provides positional tracking, allowing the player to not just look around, but move around within the virtual world. Even the smallest amount of movement helps to create the sense of presence, convincing your brain that you are actually there. After some further development using techniques normally only found in Hollywood’s special effects, we were able to render out the entire scene in 3D. We maintained the correct shape, size and scale of the scene, including all the subtle surface details, so you could enjoy the performance as if you were there.

We also added positional audio tracking. The PS VR’s binaural audio processing always provides amazing directional surround sound, but the audio in this experience now responds to proximity; move close to Joshua to hear all the detail of his violin; move back, and the natural reverb of the hall takes over.

Providing free movement around a live action VR scene is cutting edge and normally involves cumbersome camera and playback equipment. The new approach we used for the Joshua Bell VR Experience allowed us to capture the informality of a real recording session without the constraints of a special studio. It also played to the strengths of the PS4s processing capabilities. To learn more about the process, check out this behind the scenes video we captured while in development.

We’re all really happy with the results and felt that the full VR experience brought to life elements of the performance that could not be conveyed using the music on its own.

Creating this experience was an enormous technical challenge and our next steps are to make everything easier to do so that we can capture many more performances. All types of music, theatre or even sport, could benefit from this technology.

Comments are closed.

13 Comments

1 Author Reply

  • I realize this is not likely going to be done with my favorite heavy metal bands. But this is cool and i would like to see this with more artists from various genres

  • I can’t find this in the US PlayStation Store. Is this only for the EU? If so, why isn’t this available for the US? I’m reading this on the US PlayStation Blog, after all.

    • It’s there now! They sometimes publish posts here in the blog before the actual Store Update. You should always check later during the day.

  • Looks really interesting cant wait to experience it! Cant seem to find it in the US store either

  • This seems neat, and hopefully will be successful enough that many similar offerings will follow.

    And you guys ARE going to have a VR livecast of your E3 press conference, right?

    Right??? :)

    • Victoryismine52

      Oh my God this! I’ve gone to the theater to watch E3 the past few years but would love a VR experience!

  • KazeEternal

    To bad this technology requires headphones. The feel of the sound when your enveloped by it is part of the experience of being there. Headphones just don’t capture that. Still this is pretty cool. I’ll have to check it out.

    • big-_-_-_-boss69

      If you used speakers, you wouldn’t get the 3D audio effect since you would probably need like 20 speakers to get a similar experience. Remember in RE7 when you hear sounds coming from above and below you? That’s a huge factor in immersion and really helps to put you there.

  • I loved it. It was so much better then the streaming music videos they have. A full concert is something I would pay for.

  • Trophies?

  • ive always wondered why these articles dont include store links

  • This is a great idea. My first thought is this would be a fixed-point 360 degree video like many of the others, but the fact that they managed to incorporate head position as well (not just orientation) is taking it to the next level. So cool, and I hope more experiences like this follow suit.

  • This is awesome! Please make an orchestra experience!

Please Enter Your Birth Date

This content can be viewed only by people who meet the minimum age requirement.