WHAT REMAINS | WHAT BECOMES
Reimagining Museum Systems through XR + AI
Abstract
what remains | what becomes is an XR + AI-powered museum experience that invites visitors to learn from the past, collaborate in the present, and speculate on the future. The system replaces static displays with interactive stations, role-based gameplay, and future projections that show the impact of visitor choices. It creates an educational experience rooted in responsibility, reflection, and collective action.
Role: Product Designer (UI/UX/XR), Project Manager (Team of 5), Fabricator
Tools: Figma, Unity, Quest 3, Custom AI Scoring Engine, 3D Printing
Timeline: January – May 2025
Team Size: 5
Tools: Figma, Unity, Quest 3, Custom AI Scoring Engine, 3D Printing
Timeline: January – May 2025
Team Size: 5
Museums often isolate artifacts behind glass, stripping them of their rich culture and use contexts. Visitors passively observe instead of actively engaging with the systems these objects once belonged to, and might still influence today.
We asked:
How might we design interactive museum systems that allow visitors to experience history?
How might we connect historical knowledge to present-day collaboration and future consequences?
The Problem
We proposed a museum system centered around a Learn, Build, Simulate model, encouraging visitors to move beyond observation into participation, collaboration, and future thinking.
Since our initial system prototype, we created independent product prototypes for each segment to test the feasibility and educational value of this model.
The System Proposal: Learn, Build, Simulate
1. Learn: Interact with XR-enhanced, 3D-printed artifacts and collect knowledge tokens
2. Build: Apply knowledge in a cooperative multiplayer farm-building game
3. Simulate: View AI-generated long-term consequences of collective decisions
System Flow
Learn
Build
Simulate
System Prototype: Visitors would explore artifacts through XR overlays and physical interaction zones within the museum.
Product Prototype: We created a ‘looks-like’ UI prototype for Meta Quest 3 AR headsets to overlay cultural and ecological information onto artifacts. We also 3D-printed farming artifact models to assess visitor engagement and tactile interaction. Additionally, I built an architectural model to visualize the museum layout and visitor flow.
I co-led the design and testing for this segment alongside a teammate, focusing on educational journey mapping and artifact engagement strategies.
Learn
System Prototype: A cooperative multiplayer experience where visitors would use the knowledge gained in 'Learn' to collaboratively build a simulated sustainable farm.
Product Prototype: We developed a role-based cooperative game where visitors adopt roles (e.g., Crop Officer, Fertilizer Officer, Protection Officer) and unlock tools based on collected knowledge tokens. The game promotes communication, collaboration, and community-building among visitors.
Build
System Prototype: An AI-driven simulation projecting the long-term ecological and social outcomes of visitor decisions made in the 'Build' phase.
Product Prototype: We built an AI-powered prototype that generates a 50-year future projection of the collaboratively built farm, visualizing consequences such as thriving ecosystems or environmental degradation. This prototype tests how well the system can link visitor decisions to speculative futures, highlighting systems thinking.
Simulate
We centered the prototype around sustainable farming practices and artifacts from Asian cultures, choosing this theme as part of a collaborative partnership with the Asian Art Museum in San Francisco. In particular, we drew inspiration from the Dong (Kam) people, who have a long history of sustainable farming practices, notably their integrated duck-fish-rice cultivation method. This partnership allowed us to thoughtfully integrate tangible heritage and systems thinking into both our system concept and product prototypes.
Prototype Theme: Sustainable Farming in Asia
Education Design & Learning Experience
• Led the design of the "Learning" phase, where visitors explore artifacts through XR overlays and the physical 3D-printed artifact models.
• Designed and fabricated the architectural model for the proposed museum system layout
• Developed the concept map for the education journey, rooted in themes of curiosity, collective identity, and active learning.
• Designed artifact interactions to promote retention, system thinking, and transmedia engagement beyond the museum space.
Project Management
• Managed a team of 5 through design sprints, midterm deliverables, and technical prototyping milestones.
• Facilitated team alignment around storytelling, thematic clarity, and stakeholder feedback loops.
My Contribution
This project transforms passive museum visitors into learners, collaborators, and speculators. By bridging tangible heritage with immersive technology, it builds space for critical thinking, empathy, and systems literacy. It offers a participatory, future-facing museum model for an interconnected world.
Why it Matters
