Imagine a gallery where you can step into a painting, fly through the ceiling to the heavens above, or learn a dance from a virtual flamingo. It’s not a world far off. Virtual and augmented reality stand to transform the art world, and the tech incubator Oregon Story Board is on the front lines.
As a December snowstorm raged around OSB’s office in downtown Portland, the artist Kristin Lucas was dreaming of flamingos.
“I’m interested in making an augmented reality experience for the HoloLens,” she said, referencing Microsoft’s new augmented reality glasses that make it look like the digital object is a hologram floating in the room in front of you, “in which [participants] receive a dance lesson, and they’re doing a multi-species dance with a flamingo.”
First things first: She had to come up with the flamingo dance. So she watched a bunch of YouTube videos of flamingos, and then put on a motion capture suit, which has 32 sensors hooked up to a bunch of wires, and did her best interpretation.
“You are such a good flamingo,” exclaimed Thomas Wester, watching Lucas wiggle her fingers behind her hips like tail feathers.
Lucas can then animate a virtual flamingo with her dance. The idea is that a flock of strangers could don HoloLens headsets and do the flamingo dance in sync with each other and their virtual flamingo instructors, to raise awareness that few flocks of actual flamingos remain on Earth, due to habitat destruction. To viewers, their fellow dancers will have holographic flamingo head.
At this point, you may be asking, what is an artist doing at a tech incubator dancing like a flamingo? Oregon Story Board has become Portland’s de facto hub for the growing mixed reality community, which includes both virtual reality (where you wear a video headset that completely immerses you in the virtual world) and augmented reality (where a headset or something like your phone camera allows you to see virtual objects overlaid on the physical world).
“They’re very connected to the creative professionals community, but not to the visual arts community, and I thought that was a problem I could help with,” said Theo Downes-Le Guin, who focuses on digital art at his Portland gallery, Upfor.
Downes-Le Guin got to talking with one of OSB’s lead volunteers, Thomas Wester, who previously steered innovation for the interactive design studio Second Story. They came up with the idea of an artist research residency. They put out a call to artists, received more than 150 applications, and selected eight from around the globe to come in two cohorts.
“We decided to focus the residency on the idea of just becoming as familiar as you can with as many technologies as possible as an artist with no pressure whatsoever to produce something by the end of the 10 days,” Downes-Le Guin said.
Of course, most artists can’t help but make things, so each day involved guest speakers and plenty of experimentation. On the second day, they played with motion capture. The Canadian artist Freya Björg Olafson donned the Perception Neuron suit, with its 32 sensors covering her body, and copied a dance by a 4-year-old girl on YouTube. (Olafson’s previous work has played off of having professional dancers copy YouTube choreographers.) She then mapped the dance onto a 3-D avatar that bore an uncanny resemblance to Professor Xavier from the X-Men, down to the blue suit and bald head (but minus the wheelchair). In the VR headset, he perfectly mimicked her slides and arm gestures against a black backdrop, and the artists discovered that, as they moved their heads, he would seemingly dance through the air, leaving tracers of color behind him.
“I love it when he’s flying through the air, when his gesture also works with the motion,” said Lucas, trying it out and moving her head in time with his dance moves.
On the fourth day, the artists learned about photogrammetry, a technology through which they can generate 3-D digital models of physical objects by scanning them with a special camera. I found the only Portland artist in the group, Ryan Woodring, in the kitchen with Wester, using the camera to scan a clay statue he’d made of Humphrey Bogart.
A former LAIKA animator, Woodring's recent work has involved altering YouTube videos of ISIS destroying ancient sculptures or Western countries unveiling other antiquities — some of which, including the video below, are part of the exhibition "Iconoclastic" at Reed College's Cooley Gallery through Apr. 28. (Woodring will give a talk at the gallery on Feb. 24 at 6 p.m.) But he also has an interest in how we document lighter things in the digital world, like amusement park rides.
Woodring wanted to recreate the goodbye scene from “Casablanca,” as seen through You Tube videos of the Great Movie Ride at Disney World, by using motion capture to reenact the dialogue himself and then mapping it onto the digital sculptures he made using the photogrammetry, “and then kind of have these two characters in a virtual environment where they’re constantly in a state of goodbye,” he explained.
Two rooms over, I found the Brooklyn artist William Pappenheimer crawling around on the floor wearing an HTC Vive virtual reality headset.
“We’ve got three bubbles of 360 video running, and you just stick your head in one and get into the other world,” he said, as he crawled from one to the next.
Wearing an HTC Vive headset, Kristin Lucas (inset) looks into the 360 video that Brooklyn artist William Pappenheimer filmed walking across the Broadway Bridge (seen here as a 2D screenshot). She can fully move into the bubble, or explore one of the other 360 videos that Pappenheimer has placed nearby.
Courtesy Oregon Story Board
There’s a video he filmed walking across the Broadway Bridge, one shot under a bush in a garden, and one from the corner of his bathroom floor. When you put on the VR goggles and explore them, it’s kind of like being Harry Potter peering into Dumbledore’s magic basin of memories. You can look into a bubble from the edge or fully dive into it, exploring the settings, several digital animations he’s inserted like a colorful floating ribbon, and the weird ways the bubbles overlap, so that you can see two at once.
“This was a rather fantastic discovery for a way of having almost a three-part video going on," Pappenheimer said. "It’s almost a multi-narrative. You could have a whole other story happening in one. It’s sort of like three-channel video … huge potential, I think.”
In "Sky Petition City," William Pappenheimer created an app that allows users to write messages above buildings in D.C. for all to read. When someone else writes a new message, the current one is stuck onto the Washington Monument like a giant AR receipt spindle.
Courtesy of the artist
And Pappenheimer would know. He's taken part in some of the earliest AR projects as part of the group Manifest AR, which has placed virtual art in museums like Museum of Modern Art in New York that you could only see through a phone app — think the "Pokémon Go" of gallery shows. He's made an app where you can write messages in the sky over buildings that others can read, a kind of virtual graffiti. And he created a "designer drug" that you could give your phone at a party at the Whitney Museum that would overlay psychedelic patterns on everything you looked at through your camera.
Gallerist Downes-Le Guin sees augmented and virtual reality as a logical step for artists like Pappenheimer who are already pushing digital tools. But for other artists, he says it could be transformational.
“If you think of the work you make as a flat plane for the viewer to stare at, now you are given the opportunity to turn that plane into an entry point into a very flexible, very — in some ways — uncontrolled world, where the viewer drives part of the experience,” he said. “That’s a huge mental shift, from giving people a picture to look at, to a place to enter and explore.”
Of course, everyone says it still has a long way to go, particularly in terms of technology and how people and museums buy and collect virtual art. But the sky is the limit, literally — some artists are using drones.
The dance Kristin Lucas did in the motion capture suit mapped onto the digital flamingos.