About the project
Mixed Reality Storescape seeks to expand the role of the built environment inside mixed reality experiences, proposing a Context-Specific Design approach to mixed reality development that is designed for and with the architecture in which it is situated. The proposed approach allows for a thoughtful and tailored inclusion of spatial elements such as floor, ceiling, walls, furniture, and objects, posing as an alternative to the commonly adopted modular design approach to mixed reality environments, which disregards the qualities of the place and its architecture.
High-end retail spaces are designed to immerse the customer and provide plenty of opportunities for integration of media into the space.
The project is a part of the research paper “Towards Mixed Reality Architecture: A context Specific Design Approach for Mixed Reality Environments”. It lies at the intersection of extended reality (XR) technologies, architecture, and experiential retail. It explores how spatial features of a built environment can be integrated inside a mixed reality (MR) environment to create a unified real-virtual place unique to its architectural design.
Browsing Shelf
The browsing shelf was designed for four different products with virtual information mapped to the shelf and its layout extending over the wall behind it. In addition to serving as a typical retail display shelf, the shelf enabled users wearing the HMD to interact with the products in a virtual manner. The information was layered into three interactions; 1) The idle state visualised points of interest on the shelf with a translucent overlay around products, 2) Reveal on Hover made appear basic product information and acted as a signifier for further interaction. 3) Click to Expand made all product specific information visible on the wall and around the shelf.
Customization Shelf
The customization shelf facilitated a hands-free experience with highly embodied interactions, relying on hand poses . The wall-hung shelf was created to demonstrate in-store product customization in MR. On the shelf, the Virtual ingredients were placed next to their real-world counterparts. The users could grab all three of them, one by one to combine them in a virtual mixing pot which then triggered the creation of a custom product. Holding virtual ingredients allowed users to examine them from all angles. If ingredients were dropped, they would bounce to the floor and could be easily retrieved.
Wrist Cart
The Wrist Cart designed to work alongside the browsing shelf, features an ‘add to cart’ button that users could click on using the raycast. The wrist cart updates each time a new product is added. It extends the user’s body into the UI making the body an active and integral part of the UI. Tracked to the body, the wrist cart follows the user around the store space, eliminating the need to carry a shopping cart. The design anticipates separate inventories for checkout and display products and could potentially include features such as customer profiles, virtual checkout, in-store product searches, store maps, and internet access.
Augmented Ceiling- MR skylight
The augmented ceiling was designed to transform the low ceiling of the exhibition space into a virtual skylight, making the area appear larger than it actually is. This approach addresses common issues of small retail spaces such as compactness and lack of openings. It demonstrates how the physical limits of interior architecture can be virtually extended.
Context-specific Design (CSD) Approach
This project proposes a Context-specific Design (CSD) approach for Mixed Reality development, where the virtual components are specifically designed to complement the contextual features of the built environment such as furniture, walls, ceiling, windows, floor, and materials. The research investigates the advantages of a location bound MR experience. Here, the Mixed Reality Storescape actively uses four types of contexts: physical (Retail store), Temporal(activities), thematic(Bath and Body store), User(Customer and store manager).
Spatial Zoning Taxonomy for Mixed Reality Retail Environments
The spatial planning process involved creating a floor plan that considered key aspects such as interaction points, interaction types, user movement, field of view (FOV), and information retrieval. Zoning, a common architectural principle, was adapted to address the unique challenges and opportunities of Human-Computer Interaction (HCI) systems, which are often absent in traditional architecture. This adaptation led to the development of a spatial zoning taxonomy tailored for Mixed Reality Retail Environments (refer to the research document for detailed information).
The taxonomy extends beyond the design stage. It can be applied to visually demarcate areas for navigation, filter visual-spatial information, locate specific products, enable quick browsing of the space, and even personalize in-store environments based on customer profiles.
Spatial Co-location and Affordances
The shelf, a retail staple due to its versatility, retains its familiar form and function while integrating Mixed Reality UI inspired by the affordances of e-commerce websites.
Blending the virtual with the physical required establishing visual-spatial coherence through a technique called spatial co-location. Drawing from Gestalt psychology, this method integrates virtual elements with physical objects by creating meaningful relationships between them. This not only makes virtual elements appear convincingly real but also fosters new interactions, meanings, and enhances information retrieval. It goes beyond visual integration to achieve functional integration, combining physical actions—like picking up, opening, or placing an object—with familiar virtual actions such as clicking, hovering, and highlighting.
Digital Twins in Design and Development
The furniture geometries were exported as Digital Twins for providing a 1:1 scale reference to real-world furniture aiding in occlusion. The 3D models helped design the space to design for furniture orientation, user interface interactions, spatial movement and engagement within the blended environment. The exploded view illustrates how planes and volumes came together in the scene model, where the colored components doubled as Digital twins. They were modified in the game engine to include interactions, virtual objects, informative text, videos, and virtual textures.
