touchless-mac
Control your Mac with your hand through the built-in camera
Point to move the cursor, pinch to click, thumb and ring finger to scroll, wave to switch apps. touchless-mac reads your hand through the built-in camera and drives the normal macOS cursor. Everything runs on the Mac, with no network at run time. It is written in Rust, and there is a 19-minute video course on building it.
Hand tracking has to be fast enough to feel like a mouse
The camera delivers a frame every 33 ms, so the tracker has to finish well inside that window. On an Apple M5 Pro, touchless-mac takes about 9 ms to find a palm and about 1.5 ms to track a hand it has already found. It does not turn the Mac into a touchscreen: it reads your hand through the camera and moves the usual cursor.
Key Capabilities
Point and pinch
The cursor follows your index knuckle. A thumb-index pinch clicks, two quick pinches double-click, and a thumb-middle pinch right-clicks.
Scroll with thumb and ring finger
Pinch thumb to ring finger, then move. The pose was chosen from data: in recordings of ordinary pointing, "index and middle out" appeared in up to 82% of frames, thumb-to-ring in under 1%.
Wave to switch apps
An open-hand wave right or left switches apps, and a wave up opens Mission Control. When your hand leaves the frame, the cursor waits.
Fully local pipeline
MediaPipe palm detector and 21-point hand landmark model (OpenCV ONNX conversions, Apache 2.0) run through ONNX Runtime, smoothed by a One Euro filter, with enigo for mouse and keys and nokhwa for the camera.
Install
Gestures
- Point
- Moves the cursor, following the index knuckle
- Thumb-index pinch
- Click
- Two quick pinches
- Double-click
- Thumb-middle pinch
- Right-click
- Thumb-ring pinch, then move
- Scroll
- Open-hand wave right or left
- Switch apps
- Open-hand wave up
- Mission Control
Rust lessons, one per chapter
- 01
Cargo and crates
Start a project and pull in the camera, model runtime and input libraries.
- 02
struct and let mut
Model the tracked hand as data and update it frame by frame.
- 03
Option
Handle frames where no hand is found without crashing.
- 04
enum and match
Turn hand poses into named gestures and actions.
- 05
Result and ?
Report camera and model errors instead of ignoring them.
Timings were measured on an Apple M5 Pro. Version 0.1.1 targets Apple Silicon; Intel Macs are untested.
SuperLocalMemory
Local-first Memory for AI Agents
Store and recall project context across AI agent sessions through MCP or CLI, with workspace isolation, explicit operating modes and auditable retrieval.
SLM Mesh
Peer-to-peer communication layer for AI coding agents
P2P agent communication with 8 MCP tools. Agents discover each other, send messages, share state, and lock files. Works with any MCP-compatible agent. SQLite + UDS for <100ms delivery. 480 tests, 100% coverage.