Lumen: Mixed Reality

All popular AR/VR technologies today that lets you immerse relies on headsets or screens, Lumen challenges this trend and explores how people can feel immersed in their natural space by merging bits with atoms. The main inspiration behind Lumen was imagining a future with no more screens and trying to see how people will interact with the environment around them.  The research started with the question: “How might we immerse people in alternate reality experiences without isolating them in headsets?”
Several sacrificial concepts were sketched out and used for talking with experts in the realm of AR and MR. Key interactions were identified from these story sketches.  Moreover, the contributions from a LARP gamemaster through several interview and prototyping sessions were really helpful for understanding about the elements that creates believable stories. As part of research, escape rooms and museums were also studied for understanding the immersion they are capable of creating. For exploring the interactions, several different working concept prototypes were made such as for storytelling with kids, storytelling in art galleries, storytelling in homes, etc. And after testing out each of these prototypes with people, it was understood that the potential for this kind of tool lies in it being a platform. A storytelling platform that lets people tell their stories through magical interactions with their environment. And that’s how Lumen as a platform was born.

Technology behind Lumen: Lumen uses the yolo darknet machine learning platform to classify objects which is then processed by the onboard algorithm that generates stories on top of those classified objects. Narratives for this platform are designed by storytellers and game designers who will have access to the background graphical interface that serves as the story builder and dashboard. The hardware consists of a laser projector in combination with a camera and depth sensors that work to create the best projection mapping experience for the user.

For further documentation: http://arvindsanjeev.com/lumen.html

Concept video for Lumen

The process followed by Lumen can be viewed here