An Augmented Reality museum guide tailor-made to each user. It can be experienced throughout the museum or remotely at any other location.
CONTEXT
Museums with huge collections are bound to their geographical location, as well as their building size. Their curatorial layouts are far from customized to the interests of the visitors by attempting to encompass massive timespans, materials and dimensional scopes that overwhelm the visitors and leave them feeling as though their visit was incomplete. This is heightened by experiencing a sense of distance from the intimacy that is desired with particular pieces.
AIM
How might we leverage the power of Augmented Reality to enhance the way visitors experience a museum visually and tactilely?
TEAM
This project is a joint effort between Benjamin Glasser and Mario Santanilla. While we both were R&D XR Design interns at BMW in Munich, Germany, we spent much of our time in museums of all sorts.
APPROACH
A key feature of this build involved including a 24hr livestream of the gallery at all times to be displayed soley on the landing page. It was important to create a design language that wouldn't overshadow this focal point.
Feature #1
Survey
In our research, one of the key factors that led to museum disappointment was a sense of confusion and overwhelm felt when entering vast collections. Even when one goes on a guided tour, they are far from tailored to the individual's interests. Therefore, it was imperative that PointAR start with a survey to understand the user’s specific desires and interests in order to be able to structure the most precisely tailored tour.
For this, it was necessary to define a language that could easily transition to the AR experience. Through field research, we broke down the museum into three categories: time period, geography, and material form. We focused on these attributes to guide the user through the survey. This took us on a rich UI visual exploration, guided by rigorous UX research and testing.
Feature #2
Tour
When brainstorming how to improve the experience of touring through a museum, immersion was an aspect that we wanted to improve through helpful guidance. We knew we would want to focus the user's attention by removing unnecessary stimuli visitors mentioned. We would do this by removing excess noise and highlighting targeted points of interest.
An aspect of AR that excites us is the opportunity to get rid of the frame and restrictions of traditional screen borders. This is why, through various explorations on innovative guidance systems that AR affords, we landed on the inspiration of Particle Systems: a fluid tool that is malleable and adapts to each unique environment. This system helps focus the visual attention of the user while following it, aided by a voice over tour guide that immerses the spectator on the journey from piece to piece.
Feature #3
Haptic Experience
While discussing ways to improve museum immersion, we realized that we did not need to be limited to augmenting museum experience through visual cues alone. We thought specifically of old carved sculptures and that a common aspect of the modern museum experience is that users cannot interact directly with the pieces in fear of damaging the expensive, fragile works. However, this adds another layer of distancing in a museum. Our hope is to provide the user with the experience of touch to get them closer to the intimacy of what the sculptor was feeling through their creative process.
The haptic experience is where PointAR ties all of these various elements together. Inspired by experiments in particle systems, we explored Point Cloud LiDAR scanners that have become more accessible in recent years. We decided to push the limits of this tool, to record not only visual 3D assets, but to integrate them in a system of haptic rings that allows the user to feel and touch the sculptures. We prototyped a haptic set of rings that could communicate with our application to get the individual that much closer to their favorite pieces.
This also allows for PointAR to reach users that are unable to visit particular museums and still have the opportunity to experience these pieces in the comfort of their own environment to explore, feel, and study.
Feature #4
Mixed Reality
The current state of the art technology allows for novel interactions with sculptures and pieces that could not have been experienced prior. In this example, a user can see how a statue may have looked, in full color, at the time of it's inception adding new layers of depth to the museum experience
Process
The goal of this project was to explore how we could use the tools we know with AR prototyping and gain a deeper understanding of them by focusing on a possible use case in one of our areas of interests, art.
Our initial approach to the project was based on long hours of conversation we had on the current affordances of using individual vertices to structure computer generated imagery. Advancements in the usability of LiDAR Point Cloud recordings of 3D spaces and Particle Systems to simulate real life physical behaviors rid the need for the physical restrictions of the traditional frame imposed by standard screen hardware with fixed resolutions.
Our three part system consists of the survey, the tour and the haptic experience. The survey is the initial encounter the visitor has, when a series of multiple choice questions asked on their phone application determines their specific areas of interest. The walk through uses this information to create a tailored AR tour that uses a malleable Particle System to guide the user from the different locations. The haptics are activated when the user reaches a specific art piece that they want to explore further so that they can feel and interact with the piece.Using particle systems, PointAR can successfully and dynamically transition between realities and technologies. For instance, as many AR applications are currently hosted on phone apps, we felt it would be intuitive to host the survey on the phone screen which is where users currently feel most comfortable interacting with virtual buttons, but then transition seamlessly into AR space via a particle system that begins on the phone screen and transitions into the augmented space. This continues with the consistency of our visual system. Through this strategic prototyping, we created a system that could adapt in real time to have a dynamic and useful prototype to showcase and express our idea.
Using particle systems, PointAR can successfully and dynamically transition between realities and technologies. For instance, as many AR applications are currently hosted on phone apps, we felt it would be intuitive to host the survey on the phone screen which is where users currently feel most comfortable interacting with virtual buttons, but then transition seamlessly into AR space via a particle system that begins on the phone screen and transitions into the augmented space. This continues with the consistency of our visual system. Through this strategic prototyping, we created a system that could adapt in real time to have a dynamic and useful prototype to showcase and express our idea.
To prototype this guidance experience, we used Unity to create the Particle System that guides the user around the museum. This system is based on a Bezier curve that dynamically adapts to a series of objects that function as “magnets” that guide the particle system to distinct points in real time.
For the prototype we created a UI system that would allow us to manually move, rotate and activate the start, middle and end points of each Particle System section. This was necessary so that we could go into a museum environment to test this system in various spaces. By testing this, we found that these systems are successfully free of their borders and can grow, shrink, and move in space outside of screen restrictions which allows them to more properly delineate directional information within tight enclosed spaces. Furthermore, these systems that are made up of individual dots allow for the creation of digital twins of untouchable objects. This also keeps a consistent visual language throughout the museum tour.
NEXT STEPS
- One of the most important next steps that we encountered was the possibility of turning the haptic system into a much more high definition prototype to test this design concept.
- Iterate the project using other AR glasses to have a much more broader understanding of the state of the art of the AR tech.
- We are always hungry for user feedback and as the project keeps on evolving, a never ending step is to go out and user test.