NeonBeat · Ching An Yu

Overview

What is NeonBeat?

NeonBeat is a Mixed Reality rhythm game built for Meta Quest 3. Using Passthrough technology, it places a virtual game board directly on a real table in front of you. The game lives in your actual environment, not behind a screen.

Core Concept Passthrough MR places the game board on a real physical surface, merging the digital game with the player's actual room.

Design Challenge Every UI decision about color, scale, transparency, and position has to work against a real-world background that can't be controlled.

Outcome A fully playable MR rhythm game with plane detection, dual-hand input, a streak scoring system, and three iterated layout designs.

Engine Unity 6.3 LTS with Meta XR SDK. Built and shipped as a standalone Quest 3 application.

The Problem

Designing UI/UX without a screen

Designing for Mixed Reality means giving up the one thing traditional UI relies on: a controlled canvas. The background is whatever the player's room looks like: unpredictable, always changing, never the same twice. That creates problems flat-screen design never has to deal with.

01

How do you make UI elements readable against a real-world background that's always different?

02

How do you guide a player's attention in 3D space when there's no fixed screen to look at?

03

How do you give clear feedback when there's no touchscreen tap or controller vibration that players expect?

Design Challenges

Four problems to solve

Challenge 01

Readability in Passthrough

A solid UI panel would block the real environment, which defeats the whole point of MR. The challenge was keeping game elements readable no matter what's behind them.

Challenge 02

Spatial Orientation

In a 3D space, players can look in any direction. The game board needed to always appear in front of the player, anchored to a real surface, with no manual placement required.

Challenge 03

Interaction Feedback

There's no touchscreen and no controller vibration. Players needed clear visual cues to understand what they were hitting, when to hit, and whether they succeeded or missed.

Challenge 04

Visual Hierarchy in 3D

Score, combo, and judgment text had to float above the board at the right size and height to be readable from a standing position, without blocking the gameplay happening below.

Design Decisions

How I solved them

Dark Transparent Panel

Instead of a solid background, the game board uses a dark semi-transparent material. It maintains contrast for the game elements while keeping the real table visible underneath. The environment stays present without competing with the UI.

Neon Color System

Four tracks, four colors: Cyan, Magenta, Green, Yellow. Each color maps directly to a target zone, so players learn the layout through repetition, not instruction. The neon tones are bright enough to hold up against any real-world background.

Hover Feedback

When the controller points at a zone, it brightens from a dark tonal state to full saturation. Players always know where they're aiming. The change is noticeable enough to register, but not so dramatic that it pulls attention away from the incoming notes.

Tilted Track Design

Based on user feedback, the board was tilted 15 degrees. Notes spawn at the elevated far end and slide down toward the player, adding a natural sense of depth and perspective. The game went from feeling flat to feeling spatial.

Judgment Text Positioning

Score, combo, and judgment text sit above the board, scaled for a standing player. Feedback stays in the peripheral field of view, visible without pulling the player's eyes away from the action.

Neon Color System

draw here

dbl-click to clear

Yellow

#FFE600

draw here

dbl-click to clear

Green

#39FF14

draw here

dbl-click to clear

Magenta

#FF00FF

draw here

dbl-click to clear

Cyan

#00FFFF

Iteration

Three versions before the final design

The layout of the game board went through three major iterations, each addressing a problem the previous version introduced.

Iteration 01

Vertical Panel

The first version used a vertical panel standing upright on the table, like a traditional rhythm game screen. It blocked the real environment entirely and felt completely out of place in MR. The whole point of Passthrough was gone.

Iteration 02

Horizontal Flat Board

Laying the panel flat on the table felt more grounded and natural. It no longer blocked the environment, but all the notes appeared at the same distance from the player. Everything was flat, and the game lost any sense of depth or motion.

Iteration 03

Tilted Board — Final

Based on user feedback, the board was tilted 15 degrees. Notes now spawn at the far elevated end and roll toward the player, which created real depth and made the game feel grounded in the physical space rather than floating on top of it. I also brought in VFX from the Unity Asset Store, adding particle trails and hit burst effects that gave each note hit more weight and visual clarity.

Color System Iteration

The first color palette used basic red, blue, grey and yellow. After testing, the colors shifted to electric neon tones with a dark base state, making the hover and hit states clearly distinct from each other and readable against real-world surfaces.

Before

Basic red / blue / grey / yellow. Readable in isolation, but hard to distinguish during fast gameplay and too close to each other visually.

After

Electric neon tones with a dark tonal base state. Hover brightens to full saturation, making zone targeting obvious without redesigning the layout.

Features

What the game does

Plane Detection

Automatically detects the nearest real table using MRUK. No manual placement needed. The board anchors itself.

Passthrough MR

The game board appears on a real physical surface. The environment stays visible through the dark transparent panel.

Dual-hand Input

Both left and right controllers can trigger hits. Notes are distributed across four zones mapped to both hands.

Streak System

Consecutive hits multiply the score: x2, x3, x4. Missing a note resets the streak and removes the multiplier.

Hit Feedback

Perfect hit triggers a yellow flash. Good hit triggers a green flash. A miss plays a sound effect. You always know exactly what happened.

Full Game Flow

Start Screen → Gameplay → End Screen with final score. A complete loop from launch to result.

Reflection

What I learned

Designing beyond the screen

This project pushed me to think about UX in a way that screen design never demands. In MR, every decision about color, scale, position, and transparency has to hold up against a background you have zero control over. What works on a screen often breaks the second the environment changes. That forces you to design with a lot more humility.

Layering feedback

Feedback in spatial computing has to work across multiple channels at once: visual, audio, and spatial position. When gameplay moves fast, a single cue isn't enough. Players need to know what happened the moment it happens, without shifting their focus to find out.