Virtual Reality (VR) is on everyone’s lips these days and not only in the world of feature films but equally so in the world of music. Most recently we’ve had announcements and talks from the likes of Björk and Taylor Swift, but are they really grasping for more then extremely expensive videos with bouncing sounds? We have seen the introduction of commercial headsets from Samsung and, more recently, HTC and Oculus. In addition to this, Sony’s PlayStation VR is expected to land later this year. Using various cameras, lasers, accelerometers, HD screens the player’s head are tracked in 3D space, giving them the ability to both look and move around inside these worlds, and interact with objects and characters inside them. This is neither 3D movies nor 5.1 surround sound; rather, it is a wildly, undeniably a new medium in itself.
Björk is famous for her quests into the front line of music creation, i.e. her iPad album for one, and now engaging in VR. It is of course intriguing to envision how you will be attending “live” concerts in a VR environment, and over time why not streamed such events as they happen – IRL in VR.
But really the real promise of VR will come in music creation – the ability to be inside a symphony orchestra playing the violin or as a guest band member of Depeche Mode live on stage. Brower’s latest work, Playthings, can be seen as a musical virtual reality playground, and gives users the ability to play anything in their environment—including gummy bears, hamburgers, and any other ephemera.“The first time I held the Vive controllers, the first thing I wanted to do was use them like drumsticks,” Brower says.
To produce its interactive soundscape, Playthings sends MIDI data to external soft synths, which plays back musical sound inside the gameworld. Because of this, Brower isn’t actively sharing any demos of Playthings; he’s still working out some of the mechanics of the game to make it work for a home audience. Another take on VR is this one from José González where he uses a 3D-camera-bearing balloon is released and ascends to the edge of Earth’s atmosphere, to the spare and soothing accompaniment of Gonzalez’ “Every Age.” The viewer can spin around to take in the all the geography as it unfolds below.
Another favorite is this one from The Donnies. And although it look simple enough we can assure you of the complexity as also indicated by Ryan Pulliman from Specular Theory:
“The traditional tools and ways of thinking about storytelling simply don’t apply to VR. It’s is a whole new medium and language of storytelling that requires new tools, techniques, understandings and processes—creative and technical—for the way we tell stories. If what you’re trying to do can also be done in 2D, it probably won’t make for a great story in 360. We strive to let the storytelling drive the piece and adapt the approach to the given narrative. The story needs to be custom-tailored to the medium, so creators must approach the concept with a full 360-degree viewpoint in mind to enhance the story, not simply enlarge it.
In traditional media forms and at live events, there has always been a space between the audience and the content. When you strap a VR headset to someone’s face, they are immediately engaged and immersed in your story. We’re no longer spectating; we’re experiencing. Good content means allowing the viewer to feel truly transported by an experience, which comes down to attention to detail on all fronts: creative concept, resolution, audio and movement.”
Artist Ash Koosha, meanwhile, wants to integrate VR into his live shows, although this won’t involve the dystopian dream of audience in headsets. “If we want the audience to wear headsets then why are we inviting them to the venue?” he says. “If I use VR in live sets I will be doing the music inside VR headsets and people are going to see bits of it, see how I deal with the 3D environment.” VR doesn’t even have to be expensive to watch: you can buy a basic Google Cardboard set for as little as £10, which will work with your ageing smartphone, and much of the VR videos produced to date are available for free.
“To me it sounds very gimmicky to have a huge artist doing their standard, very conventional type of music in VR,” Ash Koosha explains. “Because for now, this pathway that we’re taking as artists is about a very abstract concept. It’s like transforming sound into objects and making a medium where it is totally audiovisual, you cannot say whether it is music or not. It is not even going to be called music anymore. It’s just an experience.”
In parallel we have Sony getting full into the action with Rez. The world’s first glimpse at Rez Infinite — the ultimate version of Sega’s classic psychodelic rail-shooter adventure Rez, fully remastered and evolved, including VR support and additional new content, all by members of the original development team. With “Rez Infinite, the game comes closest to what the team members saw in their heads when they were creating it: Vivid colors that blend seamlessly into one another, crystal-clear textures, and razor-sharp lines, with full 3D audio.
“I know to some people that’s all just a bunch of numbers and technical mumbo-jumbo—and in a way, I agree,” Mizuguchi says. “The only thing important about that technology for “Rez Infinite” is how it makes you feel; hopefully, it makes it easier to forget the fact that you’re sitting in front of a TV playing a game, and instead lets the real world melt away into a swirl of incredible sights and sounds that could only ever exist in your imagination.”
So what use cases are most relevant in the short to mid term perspective in VR music:
Students could learn anything from basic scales to complicated pieces, all while pretending to be playing for a packed house as their favorite artist. If nothing else, it can help keep them motivated to practice, knowing that they’re getting closer to headlining a show in real life. This is still a largely unexplored territory right now, although a number of academic papers and conference presentations have started delving into the subject.
Live music events:
VR can allow people to participate in the live music experience where they couldn’t before. It also can give artists who can’t tour (for any number of reasons) the ability to play “live,” and maybe charge admission.
If emerging artists could share virtual performances, DJs and A&R people (and fans) could have a whole new way to discover artists and give feedback.
Get inside your favorite DAW and explore all the virtual instruments from the inside 🙂