TOSHI: Gesture-Controlled Smart Lighting
A wearable that reads forearm muscle activation and controls a smart lamp with a pinch, no screen, no voice, just signal.
The premise is prosaic but the engineering is not. A forearm pinch gesture, read from surface EMG and fused with IMU orientation data, toggles a commercial smart lamp. The whole pipeline runs on a single ESP32: raw ADC at 33–50 Hz, a moving-average filter to suppress motion artefacts, an adaptive baseline tracker that follows physiological drift over a session, and a slope detector that distinguishes the fast rise of a deliberate pinch from the slow creep of postural tension.
“EMG alone fires constantly. The IMU tells you whether to care.”
A rule-based multimodal classifier fuses EMG amplitude, slope, and gyro norm through an interaction-zone state machine. The system reaches READY only when the arm is correctly oriented and motion has settled below threshold. That gating, deliberately not machine learning, cut false positives to near zero across everyday activities like typing and reaching, with gesture-to-detection latency around 120–180 ms.
Once a pinch is confirmed, the ESP32 fires an HTTP event to a local Flask gateway. The gateway logs structured gesture metadata; EMG amplitude, baseline, threshold, pitch, roll, gyro, lamp state, and issues an authenticated HTTPS command to the Tuya Cloud API. End-to-end actuation latency across the full chain ran 250-400 ms. Routing everything through the edge server keeps cloud authentication off the microcontroller and means gesture history survives a WiFi drop.