Build with Benzo Vision

Quickstart

Benzo Vision is a static site — no build tooling required. Serve the folder with any HTTP server (a secure context is required for getUserMedia):

python3 -m http.server 8080
# or
npx serve .

Open the studio at /studio.html, click Camera, and you're live. Everything runs client-side: models, inference, exports.

Vision engine

VisionEngine owns the MediaPipe Tasks lifecycle. Load a task once, then call detect() on any <video> element.

import { VisionEngine } from './js/vision/engine.js';

const engine = new VisionEngine();
await engine.setTask('hands', 'hand-landmarker', { delegate: 'GPU' });

capture.on('frame', (video, ts) => {
  const result = engine.detect(video, ts);
  // result.hands = [{ landmarks, handedness, confidence }, ...]
});

Supported tasks

• detect — EfficientDet-Lite0 / Lite2, 80 COCO classes
• pose — 33 3D landmarks, lite + full variants
• hands — 21 landmarks × 2 hands, handedness
• face — 478 landmarks for recognition signatures

Capture sources

The Capture class normalises webcam, screen share and uploaded files behind one API:

const capture = new Capture({ video: document.querySelector('#stageVideo') });
await capture.startCamera({ width: 1280, height: 720 });
capture.on('frame', (video, timestampMs) => { /* ... */ });
capture.startFrameLoop();

Face & object memory

Every saved face or object has a numeric signature stored in IndexedDB. Matching is a single Euclidean distance check against a threshold.

Face signatures

Sample every 6th landmark, centered on the nose tip, scaled by eye distance. This makes recognition robust to face position and camera distance.

function faceSignature(landmarks) {
  const nose = landmarks[1];
  const lEye = landmarks[33];
  const rEye = landmarks[263];
  const eyeDist = Math.hypot(rEye.x - lEye.x, rEye.y - lEye.y) || 0.001;
  const out = [];
  for (let i = 0; i < landmarks.length; i += 6) {
    out.push((landmarks[i].x - nose.x) / eyeDist);
    out.push((landmarks[i].y - nose.y) / eyeDist);
  }
  return out;
}

Object signatures

16×16 RGB downsample of the detected crop, normalized 0..1. Save the signature either from a live detection or from an uploaded photo.

Thresholds

The default match thresholds live in memory.thresholds:

// Stricter — fewer false positives
memory.thresholds.face = 0.45;
memory.thresholds.object = 0.18;

// Looser — catches more, more false matches
memory.thresholds.face = 0.70;
memory.thresholds.object = 0.32;

await memory.saveThresholds();

Gesture recognition

const gestures = new GestureRecognizer(bus);
bus.on('gesture', (name) => console.log('detected', name));

Recognised gestures: pinch, point, open-palm, fist, swipe-left, swipe-right, swipe-up, swipe-down.

Canvas overlay

const overlay = new Overlay(document.querySelector('#stageOverlay'));
overlay.draw(result, video, {
  showLabels: true,
  showBoxes: true,
  showPoints: true
});

Recording

const rec = new CompositeRecorder({ video, overlay, fps: 30 });
rec.start();
const blob = await rec.stop();

iOS, Android and Safari get video/mp4 (H.264). Desktop gets WebM VP9. The format is auto-picked at record time.

Model caching

const { url, fromCache } = await fetchModelCached(modelUrl);
const list = await ModelCache.list();
const bytes = await ModelCache.totalSize();

API surface

Keyboard shortcuts

• 1 — Object detection
• 2 — Pose estimation
• 3 — Hand tracking
• 4 — Face landmarks
• Space — Start / stop recording
• F — Flip camera
• Ctrl/Cmd + S — Save snapshot