Starter Hands Cohort
A bare-hands tabletop card and peg game for Meta Quest, built during Meta’s Hands-First VR Accelerator on the StarterHands hand tracking template. Spin the table with a fingertip, pinch cards and pegs, reach across the board with a hand ray, and hold a fan of cards in either hand.
Project snapshot
Meta’s StarterHands template ships fifteen demo tables that each show one hand tracking feature. This project used it as the foundation for something more specific: a working tabletop for a peg-and-card board game, played entirely with tracked hands, that behaves like a real table rather than a set of floating grabbable objects.
The game table is a Lazy Susan. Pieces resting on it turn with it; pieces on the surrounding table or the floor do not. Physics is only in charge while a piece is falling. Everything else is deliberate: pinch strength gates, release-only socket seating, a visible aim ray for remote grabs, and cards that always face the player who holds them. The demo went through ten alpha builds on Meta’s alpha channel in August 2026 and is now being ported from Meta’s Interaction SDK to Unity’s XR Interaction Toolkit as a direct, parity-checked port.
Interactions built
- Fingertip table rotation. Touch the outer rim and push. The table follows at a fixed ratio, coasts on release like a bearing, and stops on a second touch. The rim is the only touch target, so pokes never compete with pinches over the play area.
- Cards and pegs by pinch. Near pickup uses measured index-pinch strength with separate thresholds for cards and pegs. An open hand nearby shows blue, ready-to-move shows green, and a pending remote move shows orange, in a colourblind-friendly palette.
- Remote reach with a hand ray. Pieces out of reach are targeted with Meta’s aim pose and a visible ray, then pinched. Remote pegs move along the board guided by socket positions instead of jumping to the hand, with a movement-or-timeout guard against accidental pinches.
- Release-only sockets. A peg seats in a socket only after release, using a vertical capture cylinder, single-occupant rules, and occupied-socket replacement. Released outside a socket, it falls and settles on whatever it lands on.
- A full deck and a hand of cards. A 52- or 54-card deck with startup shuffle and a single exposed draw card; a discard tray that accepts a direct drop, a holder-side release, or a directed throw; five-card holders per player; and a bimanual held fan that survives hand-to-hand transfer and occlusion.
- Table and player comfort. Palm-grab corner grips raise or lower the whole table assembly. A double-fist gesture recenters the player and restores the table pose. A start seat aligns each run to the same board regardless of the headset’s remembered room position.
- Help without clutter. A native Unity 6 world-space UI Toolkit guide sits beside the table, billboards toward the player, scrolls with a ray pinch, and can be moved by its header only.
- Feedback. Pooled spatial audio cues fire once at authoritative success points: card draw, holder seat, discard, remote-move activation, and socket seating. Controllers work everywhere hands do.
Development status
- Table rotation, coast, touch-to-stop, height grips, and player reset.
- Cards and pegs with hands and controllers, near and remote.
- Guided remote peg movement, release-only seating, and occupied-socket swaps.
- Full-deck draw, discard routing, five-card holders, and the held card fan.
- World-space help guide with ray scrolling and header-only movement.
- First tabletop audio batch and a signed Meta submission workflow; alpha 0.1.10 submitted 2026-08-22.
- Direct port of the Meta Interaction SDK scene to Unity XR Interaction Toolkit 3.6, XR Hands 1.9, and OpenXR, in a separate gameplay scene with the original left intact.
- Field-by-field parity reports for pieces, pegs and sockets, cards, holders and fan, and table controls, with every serialized threshold carried across.
- Twelve dependency-ordered port phases executed; headset parity testing under way, most recently on far-ray targeting and aim-pose behaviour.
- Remaining from the Meta build: false-positive pinch diagnosis, per-object distance-ray scope, and the complete deck accounting pass.
- Target: a hand-tracking tabletop that drops into Unity’s Mixed Reality Multiplayer Tabletop template.
How it was built
Every change was driven from the headset. Each pass records the requested behaviour, the attempt, the validation, what failed on device, and the correction, in an implementation log that now runs past eight thousand lines. Scene setup is done through idempotent editor menu commands rather than hand-editing, so a builder can be rerun without duplicating components or overwriting manual tuning.
The project also served as a testbed for tooling: Unity’s command-line interface for editor automation and builds, Meta’s XR Simulator and XR Operator for autonomous input injection (useful for Meta’s own samples, not yet for this project’s controller-driven-hand path), and a scripted, signed release pipeline for Meta alpha submissions.
Media
Captures from the Meta SDK build on Quest. The board layout is the one used by Pegs & Jokers.
