Build with Benzo Vision

Quickstart

Benzo Vision is a static site — no build tooling required. Serve the folder with any HTTP server (a secure context is required for getUserMedia):

python3 -m http.server 8080
# or
npx serve .

Open the studio at /studio.html, click Camera, and you're live. Everything runs client-side: models, inference, exports.

Vision engine

VisionEngine owns the MediaPipe Tasks lifecycle. Load a task once, then call detect() on any <video> element.

import { VisionEngine } from './js/vision/engine.js';

const engine = new VisionEngine();
await engine.setTask('hands', 'hand-landmarker', { delegate: 'GPU' });

capture.on('frame', (video, ts) => {
  const result = engine.detect(video, ts);
  // result.hands = [{ landmarks, handedness, confidence }, ...]
});

Supported tasks

• detect — EfficientDet-Lite0 / Lite2, 80 COCO classes
• pose — 33 3D landmarks, lite + full variants
• hands — 21 landmarks × 2 hands, handedness
• face — 478 landmarks + 52 blendshapes

Capture sources

The Capture class normalises webcam, screen share and uploaded files behind one API:

import { Capture } from './js/vision/capture.js';

const capture = new Capture({ video: document.querySelector('#stageVideo') });
await capture.startCamera({ width: 1280, height: 720 });
capture.on('frame', (video, timestampMs) => { /* ... */ });
capture.startFrameLoop();

Gesture recognition

Feed hand landmarks into the recogniser and subscribe to discrete gestures:

import { GestureRecognizer } from './js/ui/gestures.js';

const gestures = new GestureRecognizer(bus);
bus.on('gesture', (name) => console.log('detected', name));
bus.on('gesture:move', ({ x, y, pinch }) => moveCursor(x, y));

Recognised gestures: pinch, point, open-palm, fist, swipe-left, swipe-right, swipe-up, swipe-down.

Canvas overlay

import { Overlay } from './js/render/overlay.js';

const overlay = new Overlay(document.querySelector('#stageOverlay'));
overlay.draw(result, video, { showLabels: true, showPoints: true });

Recording

import { CompositeRecorder, downloadBlob, extensionFor } from './js/render/recorder.js';

const rec = new CompositeRecorder({ video, overlay, fps: 30 });
rec.start();
const blob = await rec.stop();
downloadBlob(blob, 'clip.' + extensionFor(rec.mimeType));

Model caching

import { ModelCache, fetchModelCached } from './js/core/storage.js';

const { url, fromCache } = await fetchModelCached(modelUrl);
const list = await ModelCache.list();
const bytes = await ModelCache.totalSize();

API surface

Keyboard shortcuts

• 1 — Object detection
• 2 — Pose estimation
• 3 — Hand tracking
• 4 — Face landmarks
• Space — Start / stop recording
• F — Flip camera
• Ctrl/Cmd + S — Save snapshot