Speech Synthesis via Surface Electromyographic Control: Training Effects

Meredith Cler¹, Alfonso Nieto-Castanon², Frank H. Guenther³, Cara E. Stepp4
¹Boston University, ²Boston University, ³Boston University, 4Boston University

Individuals who use augmentative and alternative communication (AAC) devices due to motor impairment require two complementary systems to communicate: an interface that produces speech (e.g. a grid of letters on a computer screen) and an input modality with which to select targets (e.g. a head-tracker or sip-and-puff system). This study evaluated a novel AAC interface using both typical computer mouse input and a cursor control system that is available to users with severe paralysis. The novel AAC interface allows users to have direct control over their synthesized speech by enabling them to select sounds (or phonemes) rather than letters. The cursor control system translates surface electromyography (sEMG) of spared musculature into cursor movements, and was designed for individuals with spared facial muscle control as in users with high spinal injuries. An evaluation of the novel AAC interface was completed by ten healthy participants over eight training sessions. Participants produced speech with the phonemic AAC interface using both a typical mouse and the sEMG cursor control system, in order to capture how participants were learning to use the interface itself  assessed with the typical mouse control) and learning to use the sEMG cursor control and the interface. Results showed that the communication rates achieved with a typical mouse stabilized after six sessions, while users continued to improve with the facial sEMG cursor control system throughout all eight sessions. Both the novel phonemic interface and the sEMG-controlled cursor show promise for future communication applications.

Keywords: Electromyography, Speech Synthesis, AAC


Conference Sponsors