Факультет інформатики

Permanent URI for this community

Now showing 1 - 1 of 1

Decoding Speech from ECoG with Machine Translation Models
(2023) Burakov, Roman; Shvai, Nadyia; Wang, Bo
This paper explores the use and improvement of brain-computer interface (BCI)- based speech neuroprostheses, devices designed to enhance communication for individuals with speech disorders. Focusing on the machine learning aspect, we address the existing challenges associated with these systems, such as the limited vocabulary and simple algorithms of previous research and the individual variances in electrode implantation sites. Our approach reframes the decoding of speech from BCI as a machine translation problem and employs existing language models for semantic knowledge transfer. This research provides an extensive analysis of current neural speech decoding and multilingual neural machine translation methods, adapts the pre-existing M2M100 neural machine translation model for decoding ECoG data into text, and introduces a state-of-the-art model for neural speech decoding that improves upon current methods in semantic text reconstructions.