Light Painting on a VR beach

2017 – for Mystery Dish Ongoing Experiment

Light Painting (in VR and in the real world) is an ongoing pursuit for myself and Zach Becker, my creative partner in crime at Mystery Dish, our experimental AR/VR/Film studio.

If you've ever light painted before, you know that it's iterative since one can only imagine the path the light takes through the scene. You run around in the dark with a flashlight, and then peer at the camera to see if the shot turned out.

We love doing this, but for VR, we wanted to create an experience that doesn't exist in reality: slightly eerie, otherworldly, and supernatural.

This project is an ongoing labour of love, but we're sharing our process here to give people a sense of where we are headed.

Concept and Visual Target

Our goal is to makes it feel as if you are painting with light on a foggy beach at night with the stars all around you, as though you're inside a long exposure shot.

Zach created this video (Tiltbrush + Cinema4D + Octane to render) to show a possible tone and visual target, and I worked on creating a corresponding interactive VR experience in Unity.

For reference and inspiration, and to create both the rendered scene and the Unity environment, we went to Tofino, BC and shot a variety of 360 videos, panos, and timelapses, and reconstructed parts of the beach using photogrammetry.

LightPainting Photography Reference

Reconstructing a Beach with Photogrammetry

Zach took hundreds of shots of the beach to recreate a large section of Long Beach. Our photogrammetry tool of choice, RealityCapture, produces photorealistic results with an extremely high poly count. We worked with 3D artist Reece Mills to research how to bring down the size of the model and texture, while preserving its perceived quality.

Photogrammetry and Decimation Process

Drawing with Light in VR

I implemented our light-painting to be similar to what you might experience with a very basic Tilt Brush. Once I had a "flashlight/wand" painting a trail, I started exploring both how to make it look smoother/more fluid, and what else the player could do with the light once it was in the scene.

We asked ourselves when you paint, how does it affect the scene? Why do you paint—does it reveal the next element in the scene, or move a story along? Is it simply a fun mechanic that you can play with, and should it have physics? How physical is this paint?

We are currently working on some of the story-based options, but in our tests, we also looked at how it could help create the right ambience (more on that below).

Draw with Light

Creating Ambience with Light and Fog

Long Beach in Pacific Rim National Park (near Tofino, BC) is one of my favourite places, especially at moonset, when the stars come out. The shadows and fog make it feel like you are in another world. We wanted to see if we could use fog and lighting in Unity to create a similar feel in our experience.

Ambience

Additional Experiments

Light painting in Tofino is just one instance of a memory we both really enjoy. As we work on this project, we're always trying to understand better how this gets us closer to reliving a memory, or perhaps experiencing someone else's interpretation of that memory.

We are always wondering: What other things can we create that would help us revisit an event from the past, or just take us back to a happy place?

We'll share more as we learn. The road is long, and we look forward discovering it.