Home About Contact Brand Portfolio
About

Your eyes are the interface

A gaze-driven interaction model for AR — prototyped with computer vision, designed for Meta Orion.

Gaze Interaction

Designed and prototyped a gaze-driven interaction model for Meta Orion. Built a functional Python-based prototype using MediaPipe to test real-world eye-tracking latency and refine UX responsiveness.

The problem
Most AR interfaces require active manual input, creating "interaction fatigue." For always-on devices like Orion, the challenge is to move away from active clicking toward passive, gaze-driven interaction. The goal is to close the gap between seeing and doing.
The Insight
Attention is intent. In spatial computing, the first step of every action is looking. By measuring eye dwell time, we can interpret passive attention as active commitment, closing the gap between seeing an object and interacting with it.
The Solution
Gaze-driven Navigation. A hands-free interaction model that eliminates the need for physical gestures. Four core actions — scroll, pause, save, and like — are triggered entirely by where the user looks, making the interface feel like an extension of thought.
The Engineering
Building a functional Python pipeline using MediaPipe and OpenCV for real-time landmark tracking. The custom gaze-ratio algorithm features adaptive calibration, achieving high accuracy on a standard webcam without specialized hardware.

The Scroll Pattern: Natural Intent

Meta Orion glasses

I replaced manual swiping with a "Intentional Closure" trigger.
A deliberate eye closure acts as a navigation command, transforming a
physiological reflex into a functional UI gesture for a 100% hands-free experience.

By measuring the duration of gaze focus (dwell time), the system predicts
user commitment, making navigation faster than a physical touch.

Accurate gaze amidst micro-movements

AR night view

I developed a Python-based pipeline that calibrates gaze detection against
the user's facial geometry. This approach implements Head-Motion
Compensation by stabilizing the tracking relative to the bridge of the nose,
separating intentional eye movements from natural, passive head drifts.

Try the Prototype

Experience the gaze-driven interaction on your own device. Download the standalone executable or run it from the source.

Download for macOS
Download for Windows

Note: Requires a functional webcam for eye-tracking.

Requires Python 3 — don't have it? Download here (2 min install), then double-click the file. Everything else installs automatically.

Keep exploring.

Next