Text to American Sign Language

I was doing some research on VOIP products (Twilio/Vonage), and Cloud Transcription services (Google/AWS), and got curious if there are any real-time American Sign Language (ASL) generation services available on the market. I wasn't able to find any, so I started to explore what it would take to build my own real-time speech-to-sign service.

While researching and defining requirements, I learned how sentence structures differ in spoken English and ASL. For example, in spoken English, the adjective goes before the object. In ASL, the object comes first, then the adjective. This fact poses an inherent challenge in translating English to ASL because the translation can't just be literal word for word. The input sentence has to be reformatted for it to make the sense in ASL.

In theory, Natural Language Processing (NLP) and Machine Learning (ML) can be leveraged to extract meaning from English sentences. The output can be transformed into ASL in the form of video or animation. There are ASL data collections that contain the video representations for English words (ASL-LEX/SignBank/ASLLVD), but they are still quite limited.