Sign Speak

技術堆疊

React

Python

Vite

Tailwind CSS

Mediapipe

OpenCV

TypeScript

ML Model

描述

SignSpeak is an AI-powered sign language interpreter that captures hand gestures through a webcam and translates them into natural, fluent text and speech in real time. Using computer vision and a custom-trained deep learning model, SignSpeak recognizes sign language gestures, processes them on the backend, and outputs meaningful language with voice synthesis. Designed for accessibility, it works across multiple devices and supports multilingual output, helping bridge the communication gap for the speech and hearing impaired.

黑客松進展

During Hack4Bengal 4.0, we took SignSpeak from concept to a fully functional prototype. We faced several technical challenges — from unstable hand detection to gesture misclassification — which we addressed by refining our OpenCV and MediaPipe pipeline and retraining a custom CNN-LSTM model for improved accuracy. As we iterated, we moved beyond basic word-level outputs to generating fluent, sentence-based responses using lightweight NLP techniques. We also added features like a gesture confidence meter, multilingual text-to-speech, real-time feedback, and a responsive, accessibility-first UI. Through effective teamwork, rapid debugging, and continuous refinement, we successfully built a real-time sign language interpretation system that runs offline, supports multiple languages, and delivers natural, meaningful communication — a strong MVP ready for practical, real-world deployment.

Sign Speak

視頻

技術堆疊

描述

黑客松進展