An innovative professor at the National Institute for the Deaf is introducing an AI-driven tool designed to enhance the learning experience between American Sign Language (ASL) and English.
Erin Finton, a lecturer within the college’s Department of Liberal Studies, has been developing the curriculum that underlies the Grammar Laboratory AI tool for the past 15 years.
“American Sign Language is a powerful and effective medium for directly engaging students,” Finton expressed through an interpreter. “It’s the most effective language for the classroom, but currently, there’s no standardized curriculum for teaching literacy skills. This is a crucial first step, particularly for students at this age.”
The Grammar Lab features video tutorials on English grammar, presented in ASL with accompanying English captions and voiceovers. The lessons adapt to the pace of each learner.
“ASL and English are fundamentally different languages,” Finton noted. “They possess distinct word orders, grammatical structures, and syntax. While both languages express similar concepts, they do so in entirely different ways. This application is exciting because it bridges these two languages effectively.”
The AI tool generates prompts to help students hone their literacy skills and provides answers to vocabulary inquiries. This innovative solution was developed with support from Google.
“As educators, our role is to empower students to take control of their own learning,” Finton explained. “It shouldn’t be necessary for me to work one-on-one with every student for them to succeed. Having an app that allows students to engage independently is invaluable, enabling them to continue their learning journey outside the classroom.”
Sam Sepah, an NTID alumnus and an AI accessibility research program manager at Google, collaborated with Finton on this project.
“Currently, there is no standard approach or tools specifically designed for deaf and hard-of-hearing students,” Sepah stated. “While there are various AI tutoring tools available, most do not cater to the unique needs of these students.”
The app incorporates a Natively Adaptive Interface, meaning that accessibility features are integrated from the outset of the development process.
“This embodies a co-design philosophy where the design aligns with the needs of the intended users,” Sepah elaborated. “It’s not about retrofitting an existing product to suit their needs; rather, it’s about accommodating them from the very beginning.”
Finton collaborated with an engineering team to build the AI agent, as Sepah described.
“If a student struggles to understand a sentence in English or grapples with English grammar, they can ask the agent for clarification. The agent customizes its responses to the student’s reading level, helping them grasp the meaning of the English sentence,” he explained. “This exemplifies the purpose of the adaptive interface.”
Looking ahead, Finton expressed optimism about expanding the tool to include more lessons and reach a wider audience.
