OpenWearableAI
A personal investigation into whether an open, displayless wearable AI tool could be useful for electronics repair, diagnostics, robotics, and other hands-on technical work.
REFERENCE HARDWARE — Espressif ESP32-S31-Korvo-1, now running the project's own cyclops_dev firmware: camera, LC3 microphone streaming, and a dual-radio Bluetooth stack, live.
Project snapshot
OpenWearableAI began with a practical question: could a small device see approximately what I see, hear a request, and return useful guidance while both of my hands are occupied at a workbench?
I am exploring a displayless approach in which a wearable captures images, microphone audio, controls, and battery state while a phone handles applications, networking, AI services, and spoken responses. Phase 0 — structured viability testing — completed in early August 2026 with a go verdict, and the core interaction loop has since been demonstrated end to end on reference hardware: press a button, the device photographs what you are looking at, and an AI describes it aloud in the ear. The project is now in Phase 1: porting that working stack into the open MentraOS smart-glasses ecosystem.
Concept under investigation
└─▸ possible wearable (bridge module + clip-on arms)
└─▸ iPhone — forked MentraOS app
└─▸ AI service or application
└─▸ spoken response
The present mechanical idea places a centered camera and ESP32-S31 in a module above existing eyewear. Clip-on arms might carry batteries, speakers, and possibly microphones. This arrangement has been documented because it is worth testing—not because it has been proven comfortable or practical.
The phone layer is no longer hypothetical: the project maintains a forked MentraOS iPhone app, and the firmware implements MentraOS’s published OEM Bluetooth specification for third-party glasses—so Cyclops presents itself to the phone the same way first-party hardware does.
- Cyclops — a provisional name for one possible first glasses-based system experiment.
- WCM-1 — a longer-term idea for extracting a carrier-agnostic compute module if the earlier experiments justify it.
Investigation status
- Phase 0 viability tests passed (August 2026): a 28-photo, three-model vision panel; a measured readable-marking floor of ≈ 7 px character height at a natural 28–32 cm working distance; and the hearing-aid audio path confirmed through daily use.
- Model choice proved to be a safety decision: in blind runs, one model made zero hallucinations while the other two overrode printed part numbers with what their training expected to see.
- Custom firmware (cyclops_dev) runs the camera, microphones, and buttons on the Korvo-1, with A2DP audio and BLE data verified simultaneously on the one chip.
- 720p photos reach the iPhone over BLE — about 3 seconds on the first transport, near-instant after moving to the faster channel Mentra’s own hardware uses. Live LC3 microphone streaming works on-target.
- The full loop — button press → photo → vision model → spoken description — was demonstrated end to end on 2026-08-09.
- Cyclops pairs as a first-class device in the forked MentraOS app (2026-08-20): it appears in the real pairing flow and lands on the home screen as the connected glasses.
- Final firmware round-trip verification against the MentraOS app (a message-framing fix is flashed and boot-verified; the phone-side confirmation is next).
- Wiring photo delivery into MentraOS’s own photo-request flow.
- An acceptance run of the vision loop through the wearable itself.
- Speaker output on the hardware — Phase 0 promoted arm speakers to a first-class output path, but they remain untested.
- Thermal limits, battery life, and mechanical comfort — unanswerable until a wearable prototype exists.
- Whether the overall idea deserves development beyond reference hardware.
What has been tested
The experiments this page once listed as plans were run, and they passed. Vision models were compared on 28 real workbench photographs; the striking result was not average quality but variance — on unfamiliar part numbers, two of the three models silently substituted what their training data expected over what was actually printed on the chip, while the third made no such errors. That finding shaped the product requirements (honesty prompting, capture-then-crop, retry at a new angle) and was later reproduced live through the wearable loop itself. The smallest-readable-marking floor was measured at roughly 7 pixels of character height, the natural lean-in distance at 28–32 cm, and the hearing-aid audio path was validated in daily use — with wearing comfort, not audio quality, emerging as the weak point.
Phase 1 then moved past the factory demo entirely: project firmware now drives the camera, microphones, and buttons; photos travel to the iPhone over Bluetooth in under a second on the same transport Mentra’s own glasses use; and the complete loop — press a button on the hardware, hear an AI describe what it photographed — works end to end. The firmware speaks MentraOS’s published protocol for third-party glasses, and the board now pairs inside the forked MentraOS app as if it were first-party hardware.
What’s next: closing the final firmware round-trip verification, delivering photos through MentraOS’s own request flow, an acceptance run of the vision loop, a speaker-output hardware test, and then the first step beyond reference hardware — mechanical prototyping of the clip-on form factor.