The goal of the Abilities Hackathon is to use technology to solve and address problems for those with disabilities or challenges. Our group learned that some Deaf people may prefer to view someone signing instead of reading long passages of text in English (such as an online news article). This is because English is likely a second language to them while American Sign Language (ASL) is their first language, and therefore more natural to them. Our group decided to build a "text to ASL" interpreter. A browser plugin would allow the user to highlight any text on a web page, right click on it, and select the option to "sign this". The plugin would then display a video of an interpreter signing the text that had been selected in ASL. The translating and video creation was done automatically by our server. The server would parse the provided text one word at a time and search an online ASL dictionary for each word. The online dictionary would provide us with short videos of individual words in ASL. We would then stitch together all of the individual videos to recreate the sentence(s) that had been selected, and then display that final video back to the user. We also made a mobile Android app that worked in a similar fashion. Pretty simple, right? Well, while it looked cool, we learned that it wasn't going to be very practical yet. We talked to multiple ASL interpreters who were quick to point out that translating English to ASL was not that simple. They explained that ASL has a completely different sentence structure compared to English, so word order and grammar is very different. Clearly our naive word-to-word translation would not be enough. We didn't have the time or the expertise to handle all of that. To make this tool actually useful would take much longer than a weekend. Nonetheless, we felt good about the foundation we had laid for future work in this area. We had a lot of fun working with each other, and learned a lot of about ASL, Deaf culture, and language processing. Handy was highlighted as a Staff Pick on devpost.com.
Also worth noting is that the Abilities Hackathon was later featured in a mini documentary series, Make It Work, on Amazon! You can spot me working in the background throughout Episode 2 on Amazon Prime!