February 21, 2023

Beyond 2D: A Love Affair with Volumetric Video

‍In 2022 Michelle penned an article on her passion for Volumetric Video for LBB. You can read the article on LBB here. It was also featured in LBB's Production Power Moves of 2022 recap.

Michelle has been lucky to be involved in several Volumetric shoots over the years. The Antipodean has an extensive roster of production partners from stages, to directors, to delivery pipelines to creative production companies for the AR, VR and immersive web/content experiences. Reach out today with your Volumetric questions or needs.

Read the article in full here...

Michelle Excell, the founder of The Antipodean, creative tech consultants, fell in love with volumetric video when she put on a VR headset at Lytro’s office in 2016 and watched a demo of what volumetric would become.

"You're capturing them from all angles with standard cameras, but also infrared cameras that capture depth. You are literally recording their volume. What you are actually getting is a 3D video file of that subject — talking, moving, dancing."

To further discuss her love affair with volumetric video, see examples of it in action, and highlight how this groundbreaking tech can elevate a brand's presence in the market, we sat down with Michelle.

"It was amazing and hilarious. It was a short faux-recreation of Kubrick’s faked moon landing," Michelle says."It felt like a standard 360 video experience at first, but they’d shot the actor playing Kubrick with their crazy new camera rig. The Lytro team member told me to lean in, and I actually got closer to the actor. It was my first experience of real video with partial 6dof, and it seriously blew my mind!"

So, what exactly is volumetric video?

Volumetric video is a production technique involving cameras that surround and capture subjects to create a 3D moving image that can then be viewed from any angle.

"What you end up with is the re-materialising of the subject in 3D. You're capturing them from all angles with standard cameras, but also infrared cameras that capture depth. You are literally recording their volume. What you are actually getting is a 3D video file of that subject — talking, moving, dancing," Michelle says.

The volumetric capture can then be used for campaigns using augmented reality, virtual reality, holographic displays, and even web and traditional video production. As a result, and given the right planning, the technique is highly flexible.

[VIDEO OF HANNAH FROM MICROSOFT]

Above: A Microsoft Mixed Reality Capture Studios representative talks about volumetric video in Michelle’s living room.

‍How can volumetric video be used?

Michelle thinks some of the most creative volumetric work has been from the music industry.

"Whether it's a big name and studio, or an indie band and creator, some of the best work I've seen in this space is from music videos and special projects. Maybe it's got to do with the performative nature of their work, or the creators they collaborate with, but they've figured out some really incredible ways to work with it"

Michelle remembers the 2018 mixed-reality campaign by AMEX Music featuring Justin Timberlake being a stand-out. "Outside In" used volumetric video to allow Justin to come to life in your living room, launched through the AMEX music app. You could explore a song from his album as elements of it unfold around you, and you could also shop for merch.

"Having a life-size, hyper-realistic three dimensional Justin Timberlake appear in your living room - chatting away -is surreal and oh-so-memorable. Whether you're a fan or not."

Above: Outside In, from American Express Music and RadicalMedia, took users inside the creative mind of Justin Timberlake (and into his new song) thanks to volumetric video.

LA-based volumetric capture studio, Metastage — who Michelle highly recommends working with — has compiled a helpful list of popular use cases for volumetric video. They are quick to remind us, though, that this only represents the tip of the iceberg:

Sports - bring sports fans closer to their favorite athletes.
Education - bring the world's best teacher into your living room or classroom.
Training - create interactive training scenarios or simulations.
Virtual tours - create a realistic experience with an expert guide.
Trade shows - bring your lead executives with you to present at trade shows.
Museums - use holographic capture to bring installations to life.
Inspirational - create connections between your customer and inspirational figures.
Games - create celebrity cameos to elevate video games.

“If you’re a brand and you have a known and loved celebrity or ambassador for your campaign, you’ve solved the hardest question: ‘will people engage with it?’ Then you just need to make darn sure you make something special, and not just rely on that star power.”

"But look, it doesn't have to be what we would deem 'high quality' to be good. I see so many music video creators and directors playing with volumetric. They don't overcomplicate it; they get straight to the essence of what makes it unique and compelling and don't obsess over photo realism," Michelle explains. "There are two projects that stand out for me from the last year: ASAP Ferg — Green Juice. and Caoilfhionn Rose — Fireflies."

An early example of a project Michelle was involved with is a powerful PSA made by VISYON and Left Field Labs for RK Venture and New Mexico Department of Transportation. It’s an immersive VR experience about the risks of drunk driving, delivered via a reenactment from the perspectives of the drunk driver, the victim in the accident, and the responding officer. It was shot at Microsoft's Mixed Reality Capture Studio in San Francisco, and highlights how PSAs can be even more impactful when the immersion of VR is coupled with the realism of volumetric video.

"You completely captured all the emotions and pain I experienced, truthfully and accurately.”

- A real DWI victim commenting on New Mexico Department of Transportation’s experience.

The fashion industry was an early adopter of the technology, along with the music industry. Holograms were initially all the rage, but we’re now seeing practical prototypes beginning to emerge. These include a Holographic telepresence test in association with Telefónica, shot with Evercoast and premiered at MWC2022.

"Like most immersive tech, you need to see it and feel it, to 'get it'. Then it becomes a case of, 'Oh my god, I need this! How can we use this for our next campaign/installation/app?'. I don't think many other technologies have that immediate emotional effect."

Michelle's love for new and exciting creative tools spans the immersive industry.

"My love affair is really with all things spatial because I'd be remiss not to mention volumetric video's cousin, photogrammetry, and you can't have volumetric video without spatial audio."

As the name suggests, photogrammetry involves taking (a lot of) photographs of an object or scene from all angles and stitching them together to create a 3D model. Photogrammetry has many applications and is used commonly in mapping, architecture, engineering, archaeology, and increasingly in retail.

Day 336: Wooden armchair by Emm on Sketchfab

Who is using volumetric video?

As we have learned from Michelle, music is one of the biggest industries benefiting from volumetric video, but some of the world's more prominent brands have started to utilize the technology for advertising, product services and even customer services.

Balenciaga completed their real-time Unreal Engine experience Afterworld with volumetric video models for their Fall 2021 collection.
Nike's Virtual View allows customers to try on their products virtually before purchasing. Being WebAR makes it super accessible.
“Jadu is an app I’ve been promoting for years”, says Michelle. It allows fans to perform alongside holograms of their favorite musicians and performers, and they’ve recently expanded into NFT AR collections. “I’m a huge fan!”
Burger King created a WebAR experience that allowed customers to experience a private AR performance by Tinie on a Whopper burger.
The New York Times Magazine created an augmented reality experience that brings a hologram of model Ashley Graham into their living room.

"I was so excited to see volumetric capture show up in SpaceJam. As a technologist, you can love a tech all you want, but others have to realize its potential and embrace it with you. You have a spidey sense for when techs are about to go mainstream or reach a tipping point, when you see them ‘in the wild’ and not just in our bubble. So when I saw capture-tech show up in SpaceJam last year, I knew millions could easily grasp the idea."

To keep her finger on the pulse with these advancements in tech and to connect with her ever-expanding network, Michelle invests a lot of her time with tech groups and communities and even hosts workshops and speaking events that highlight how immersive technologies are for every brand.

"I went to a special Volumetric Video AWE Night NYC event in 2018 with Intel, Metastage, NY Times, Google and Scatter. It was an amazing intro to the companies on the cutting edge in the early days, companies I would later have the opportunity to work with."

Michelle stays active in volumetric community groups and in regular contact with her partners at capture stages and supporting tech platforms such as Arcturus, the makers of HoloSuite (an industry-leading tool for powering volumetric experiences). Arcturus also shine a light on the tech’s growth by publishing a global capture stage map every quarter.

Diving Into Immersion

One thing is clear: Viewers expect more than 2D experiences now, and consumers value brands that embrace new technologies. In this rapidly changing digitized world, businesses need to meet demands and find new ways of producing content to keep up with the competition.

"Volumetric video allows you to create experiences you simply can't make any other way — a prerequisite for me to be an advocate for a tech. We almost got through the whole interview without saying 'holograms’, but low or high fidelity, the feeling of presence is real. It's live theatre. It brings digital into the real world. It puts you into the action."

"You don't need to understand how it works - we can take care of that - you just need to feel the potential of it, to feel that magic and know instinctively there's a reason for using it.”

“I still have 'a-ha’ moments. On that shoot at Microsoft's studio, between takes watching playback on the monitor, the director reached out and rotated the volumetric clip of an actor to check his pants weren't causing issues with the capture when he crouched. I keep that clip on my phone to show people why volumetric still gives me goosebumps."

With a clear passion for volumetric video, Michelle Excell sees an incredible opportunity and untapped potential in how brands and creators of all kinds can take advantage of this unique and powerful media tool.

Michelle's creative technology consultancy, The Antipodean, works with brands and agencies to unlock the potential of new technologies. She understands that immersive technology can be overwhelming, so she works with her clients to create unique strategies to implement the right tools for their business.