SignBridge at the CCIR Winter Symposium

In December 2025 we presented SignBridge at the CCIR Winter Symposium, with the abstract appearing in the conference proceedings. SignBridge is the research project behind Gesturely — the engine that turns American Sign Language into English while trying to keep what the signer actually meant.

The problem we set out to address

Around 70 million Deaf people worldwide rely on sign languages, yet very little digital content is accurately captioned. Automated systems tend to stumble on continuous, dynamic signing — producing output that is grammatically off and strips away the signer's intended meaning rather than preserving it. ASL and English are two grammatically distinct languages, so a word-for-word mapping is not enough.

What SignBridge does

SignBridge pairs high-density pose estimation with a large language model that chooses between multiple candidate signs based on sentence-level meaning, not individual word-confidence alone. We call this coherence-based selection. Concretely, the system:

• extends the OpenHands model to OpenHands-HD, tracking 83 body points (up from 27) including detailed finger movement;
• applies 50× pose-data augmentation for training diversity, and uses a transformer to produce Top-K sign predictions; and
• asks Google's Gemini (gemini-2.0-flash) to recover meaning by selecting the semantically appropriate sign from those Top-K candidates.

Results on WLASL-100

On the WLASL-100 benchmark we measured statistically significant improvements across every metric:

• word-level accuracy rose from a 72% benchmark to 80.97% Top-1 and 91.62% Top-3;
• sentence-level grammatical plausibility rose from 39 to 75 (p < 0.001);
• semantic content preservation improved from 75 to 88 (p < 0.001).

Measuring translation quality: CTQI

Because there was no good single measure of whether a sign translation worked, we introduced the Composite Translation Quality Index (CTQI) — a prerequisite-chain metric in which sign recognition, content fidelity, and output plausibility all have to succeed. CTQI improved from 43 to 69 (p < 0.001, Cohen's d = 1.04, a large effect), with 82% of test sentences showing improvement.

Where it is going

SignBridge shows that AI can bridge two grammatically incompatible languages in real time on ordinary consumer hardware. That foundation is what Gesturely is built on, and it is now growing into a tool for classrooms — a curriculum companion for students and teachers learning ASL, alongside the live conversation and service-counter uses. More on that soon.