Accessibility × Computer Vision

Sign Recognition,
End to End

Collect images, train a classifier, inspect model analytics, and serve real-time browser inference — all from a single project. One command gets you running.

View on GitHub API Reference

# clone the repo
git clone https://github.com/Life-Experimentalist/SignLanguageDetector
cd SignLanguageDetector

# uv installs Python 3.12 + all deps automatically
uv run app
▶  http://localhost:5000

# train a model interactively
uv run train

# headless REST API — no browser needed
curl -F "[email protected]" http://localhost:5000/api/predict

Project Stats

Total Views

Loading…

Counter Status

Checking…

What This Project Gives You

Real-Time Browser Inference

Webcam frames go from the browser to MediaPipe landmark extraction to a trained classifier — all without a plugin. Works the same whether the server is local or remote.

Interactive Training CLI

uv run train walks you through image collection, dataset creation, classifier training, and model analysis in a guided step-by-step flow.

Headless REST API

POST /api/predict accepts a file upload or base64 JSON and returns a prediction, brightness, and contrast — no browser, no session, pure HTTP.

Model Analytics Sidebar

Every trained model ships with a JSON report. The sidebar shows accuracy, top-3 classes by F1, the weakest class, and weighted averages at a glance.

Hot Model Switching

Switch between trained .pkl models from the UI dropdown without restarting the server. The new model loads instantly via GET /select_model/<name>.

One-Command Setup

uv run app provisions Python 3.12, creates the virtual environment, installs all dependencies, and starts the server — in a single command.

End-to-End Pipeline

From raw webcam captures to a live inference server — every stage is automated.

Setup

Install & Run

Install uv once, then a single command handles everything else — Python version, venv, and all pip packages.

uv run app

Step 1

Collect Images

Capture webcam frames per sign class. Blurry frames and frames without a detectable hand are skipped automatically.

uv run train
# choose Stage 1 in the menu

Step 2

Build the Dataset

Extract normalized MediaPipe hand-landmark coordinates from each image. The dataset is saved as a pickle file ready for training.

uv run train
# choose Stage 2 in the menu

Step 3

Train the Classifier

Train a RandomForest classifier. The model, a text classification report, and a JSON metrics file are all written to models/.

uv run train
# choose Stage 3 in the menu

Step 4

Generate Reports

Convert text reports to structured JSON. The sidebar loads these to display per-class F1 scores, accuracy, and aggregate averages.

uv run convert-reports

Step 5

Serve & Infer

Open the browser for live webcam inference, switch models from the dropdown, or call the REST API from any HTTP client.

uv run app
# → http://localhost:5000
# → POST /api/predict

API Quick Reference

Send an image, receive a prediction. No session or browser required.

Multipart

File Upload

curl -X POST http://localhost:5000/api/predict \
  -F "[email protected]" \
  -F "show_landmarks=false"

JSON

Base64 Image

curl -X POST http://localhost:5000/api/predict \
  -H "Content-Type: application/json" \
  -d '{"image_base64":"<base64>"}'

Response

JSON Payload

{
  "prediction": "A",
  "brightness": 142.5,
  "contrast": 68.3,
  "low_brightness": false,
  "model": "model.pkl"
}

Sign Recognition,End to End