Developer Insights
GAF used in creation of VR App - The Bohemian Rhapsody Experience
Developer: Enosis VR, Google Play
Location: Los Angeles, CA
Game Title: Bohemian Rhapsody Experience
Game Genre: Launch, Arcade
By Vangelis Lympouridis, founder of Enosis
We set out to create a virtual reality app in collaboration with Google Play and Queen called The Bohemian Rhapsody Experience. It offers a journey through Queen frontman Freddie Mercury’s subconscious mind and recreates the sensation of being onstage with the band, with visual and audio elements that respond to the user's movements.
We needed to develop a new pipeline for our Bohemian Rhapsody Experience application. After much research and trial and error, we decided the most productive approach would be to accommodate the use of Flash and GAF together. We made certain wrappers and systems to make the most of it and the resulting pipeline turned out to be very productive. We used it in many animated sequences that were produced in Adobe Flash and then exported to our app using GAF Converter.
We have tested numerous technologies to port animations to mobile and GAF was the most robust and performed much better than anything else. We were trying to optimize processing power in any instances: CPU, GPU, Memory and when we found GAF we were amazed by optimizations we could do. Our application features 6 minutes of full VR experience in just 160Mb. Any other similar application out there would probably be more than 1Gb relying on heavy videos. Of course, there is always room for improvement and our developers are looking forward for optimizations in GAF to reduce number of drawcalls in newer versions.
Our team was under very strict schedule and didn’t have time to talk to GAF support at all, so we got all the answers by exploring GAF as we went along. Bohemian Rhapsody first beta was done in unbelievable 7 weeks! The 2D and 3D animation in The Bohemian Rhapsody Experience was modelled and animated by a small army of animators using Maya. We had 6 engineers, 1 lead engineer, 8 in-house artists and around 20 other artists all over the world working on the project.
Bohemian Rhapsody is made up of multiple parts that greatly vary in style and tempo. Billed as a creative exploration of beloved singer Freddie Mercury’s subconscious mind, the VR experience transcends between styles. At one point you’ll join a cartoon version of the band on stage to rock out, while in other moments you’ll observe motion captured 3D dancers and surreal landscapes.
For our project we used a combination of different techniques. In VR one of the most interesting moments is when something reaches out to you or when you see something very close. 3D objects are the best for this kind of experience and we used various 3D software to develop these animations. Also, we used 3D motion capture methods to come up with the beautiful dancing silhouettes, while GAF served the purpose of providing many moving details and fusing interesting 2D elements on the screen.
One of the key problems in our project was audio synchronicity, but this issue was caused by Flash, not GAF. We didn’t know this at the time but if you give Flash 48kHz sound it automatically changes it to 44.1KHz. Our sound was originally produced for interaction purposes using Dolby’s Audio Technology at 48kHz and because of the issue with Flash our animation sequences was sometimes slightly out of sync and we tried to correct this with GAF by changing FPS but we couldn’t maintain decimals in the timeline. So in the end we had to correct the out of sync points manually. We worked with Dolby Laboratories to remix Bohemian Rhapsody’s original music stems using the most advanced immersive audio techniques to date, creating an unparalleled audio experience with spatial sound. Not only can the viewer enjoy a true surround mix, but the music truly comes alive as they turn their heads to follow the action. So now you have a chance to experience Queen’s timeless track in a completely new, immersive way. The immersive 3D, 360-degree journey utilizes interactive elements and spatial sound, allowing users to step inside the song.
For our company, animation is a tool and while for some music videos it may not be appropriate, we truly believe that in the next few years the low-end VR market (VR for Mobile phones) will grow substantially and our team will have an advantage with our pipeline being ready to provide any VR experience. Talking to other animation studios in LA I see that there is a lot of room to create appealing VR experiences using GAF.
Here is a behind the scenes video about the making of the Bohemian Rhapsody VR app:
This VR app is a is a testament to how engaging and innovative VR can be. Most of the interactive experience is triggered by the viewer's gaze. As the user engages with the app, varied elements pop up in front of their eyes. "The user doesn't necessarily sense that he's triggering the elements popping up, but in fact that's exactly how they occur. Where their attention is, that’s where action unfolds,” Vangelis Lympouridis, founder of Enosis.