Stony Brook University SUNY Korea

More than 70 million people worldwide suffer from stuttering. Without intervention, many of these individuals endure this communication disorder for their entire lives. Even so, intervention therapies may not be accessible as they can be considered out-of-pocket expenses and not always covered by insurance. In a lifetime, the stuttering population is disproportionately at risk for heightened anxiety, insecurity and isolation. This life-long speech disorder stigmatizes its victims as socially inept. Personally and professionally, these individuals are limited by a fluency issue that is out of their control.

Today, thanks to Artificial Intelligence (AI) and Stony Brook University faculty, people who stutter have the opportunity to build confidence through speech improvement.

Stony Brook University PhD candidate Bhavya Ghai and Professor Klaus Mueller from the Department of Computer Science, have proposed an AI-based script-writing tool: Fluent.

Fluent is the first of its kind. Much of the literature that addresses the intersection of stuttering and AI focuses on a singular facet — stuttering detection. While there is no cure to stuttering, only intervention, this literature looks at diagnosis, not solutions. Fluent addresses the latter.

Through the use of AI, the smart writing tool leverages speech patterns of people that stutter, specifically substitution tendencies. Through this, Fluent creates AI-driven inroads for continuous speech.

Those that stutter engage in many practices to conceal their disfluent speech such as avoiding communication altogether, the use of fillers (“um,” “like,” “so”) and substitution. Speakers enact substitution to replace trigger words with appropriate synonyms. These phonological patterns do not quell stuttering, but they mask inconsistencies in speech. Fluent strives to make these substitutions easier.

Fluent is powered by Active learning (AL), a specialized branch of machine learning (ML). AL is responsible for training ML models through user feedback. Queries prompted by AL aptly determine phonetic sequences that induce stuttering episodes.

“In our context, active learning expedites the process of learning an individual’s unique condition, i.e., the phonetic pattern they might struggle to pronounce,” said Ghai.

Upon starting up Fluent, users are asked to tap into their challenges with stuttering. Fluent immediately asks for user feedback by prompting the user to annotate at minimum five words with their level of difficulty to pronounce through a binary: “easy” or “difficult.” As the inventory of words increases, with the corresponding personalized-annotations, Fluent better adapts to specific user needs.

Through these inquiries, Fluent posits trigger words and replaces them. Synonyms are harnessed from an online database that covers multiple online dictionaries: DataMuse. Accessing a single online thesaurus is reputably unreliable, especially in the case of people who stutter, where certain words are discarded for their pronunciation limitations. The variable index of synonyms provides an expansive phonetic opportunity for Fluent users. These replacements appear in a drop-down list when the user hovers over a highlighted word (which indicates possible difficulty in pronunciation). Users then click on the word that best befits their speech.

Post-evaluation, Fluent expresses a positive trend of accuracy. Fluent’s classifier reports an 80 percent accuracy with limited user interactions. As these interactions increase, so does its accuracy.

Design advancements can be employed to address more subtle and nuanced speech disfluencies. AL has a technological edge because the consistent retraining of the ML model allows for more refined feedback. However, detecting trigger words is not the only solution Fluent provides. Stuttering episodes are not only produced by specific phonological patterns, but also situations, speakers and so on. Future endeavors with Fluent set out to explore the intricacies of situations that can spur a stuttering episode in longitudinal and situational contexts.

“In the long run, we hope that technologies such as Fluent might be integrated as an accessibility feature into popular tools like MS PowerPoint, MS Word, Google Sheets, etc. for a broader impact,” said Ghai.

— Alyssa Dey