The power of immersive journalism

I’m Nonny de la Peña. I pioneered the first VR piece ever at Sundance, particularly looking at, how do we use virtual reality for nongaming. Important stories, like Hunger in Los Angeles, which actually put you on the scene of a food bank, when a man with diabetes who didn’t get food in time, his blood sugar dropped too low and he collapsed into a coma. And that piece premiered at Sundance back in 2012. And after I made that piece, I realised that this was a medium that really could scale and become a very important part of storytelling.

It also really inspired a lot of other individuals to embrace this as a space to do storytelling, most famously Alejandro González Iñárritu, the Oscar award-winning director of Revenant, Birdman. He saw Hunger, and it led to a piece which I was lucky enough to work with him on, Carne y Arena. Many of the people have told me since then that either Hunger or the next piece I did, Project Syria, about Syrian refugees, that these pieces have continued to inspire many individuals to embrace this as a place to do important storytelling.

The very first piece I ever made was in partnership with a digital artist named Peggy Weil. I’d been a doc filmmaker, and I’d made a piece called Unconstitutional that had a big section on Guantanamo Bay prison. And the prison was still open, as it’s still open today. And I felt like we just weren’t really paying attention to the lack of habeas corpus rights of the people we were locking up there. And it was a really important journalistic story. And yet journalists weren’t allowed, or anybody, to go and see what was going on. So along with Peggy Weil, we applied for a grant funded by America’s MacArthur Foundation.

And we won, and were brought to Bay Area Video Coalition to come and build a virtual Guantanamo Bay prison Second Life. And Second Life is a persistent virtual world. We really tried to be as accurate as possible. I used video that we had. We reconstructed Camp X-Ray and then Camp Delta. And people could walk around these spaces that were otherwise off limits. And that really attracted a lot of universities to study it. And it got a lot of attention. And once the virtual Gitmo was built, I started thinking about how this could be used for all kinds of journalism stories. I’d been a correspondent for Newsweek. I’d written for New York Times, LA Times, major, major news organisations.

I’d worked for the BBC. I then took all that knowledge and really brought it to bear in my thinking about, how do you do embodied spatial storytelling, and began this journey that led to the “Immersive Journalism” paper. I was invited to the lab of Mel Slater and Maria Sanchez-Vives in Barcelona. And using all the Freedom of Information Act material I had on Gitmo, we built a piece that put you in the body of a detainee in a stress position. And that was an extraordinary moment for me. Because we asked people to sort of sit up in a chair like I’m sitting now, but with just their hands behind their back and they were wearing a breathing strap.

But what happened was, you saw yourself in a virtual mirror, hunched over with your hands locked behind your back. And we’d say, OK, what was your body like when you were doing that piece? And they’d consistently report that they were hunched over when they were not. And that was when I was like, wow, this is really powerful stuff. And we published a piece in the MIT journal Presence about that. And it was the first time that we really started talking about immersive journalism.

One of the things that’s very different about virtual reality journalism, immersive journalism, is that it puts you on scene in real stories. It makes you feel like you’re actually there. And it really closes the distance, the gap of a 2D video that you’re watching, kind of peering like a spectator, versus the sensation of, oh, my god, as my son once put it, this feels like this could happen to me, too. I’m here. And I think that really has a different impact in the way that individuals experience a story.

We’ve done some studies looking at taking a piece we did on solitary confinement, with America’s premiere investigative documentary series Frontline, and then also with NOVA, which is our science programme on climate change. And we did some studies on looking at it- we took the same pieces and we made just flat, regular screen-sized videos. We made 360 videos. And we also had are real immersive experience. And by far, people reported the immersive experience was the one they connected with the most, was the most impactful for them. It’s just a really different experience. It’s a really different feeling. That isn’t to say that 2D news or radio or- obviously, podcasts are huge now. Radio has not gone away.

Similarly, regular journalism will not go away. But this is an important part of the new way that we’re going to be telling stories.

I think the main thing that is a cohesive element and is going to be uniform throughout everything is, you have to think, what does it mean to be embodied in the story? What is my audience going to feel as an embodied viewer? And I always say to people, before you start, shut your eyes and put yourself in the middle of this. What is it going to feel like if you were there? And I think that’s the number one point that people have to begin with when they start thinking, I’m going to do an immersive story, whether it’s fictional or nonfictional narrative.

In Fortnite, in Minecraft, in any of these games, people are growing up feeling that they have a digital presence, they have a spatial sense of being in this space. And there’s just a lot of research that what you see yourself, your avatar, you connect to it, even in this small digital way. And with virtual reality, we take it to a much more expansive level. More and more, the controllers, you see your hands. You have this sense of your body being there. That is only going to increase. And I think that this is a really important principle. What does it mean to have your audience feel like their body is inside the story?