Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Improving Robustness of Real-Time Speech Detection from Chronic ECoG using Targeted Data Augmentation
Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai
Camil Ziane1, Mohamed Baha Ben Ticha1, Guillaume Saldanha1, Clément Arvis1, Xingchen Ran1,2, Amina Fontanell1, Karteek Alahari4, Thomas Costecalde3, Lucas Struber3, Serpil Karakas3, Shaomin Zhang2, Guillaume Charvet3, Stéphan Chabardès1,3, Blaise Yvert1; 1Univ. Grenoble Alpes, Inserm, U1216 Grenoble Institut Neurosciences, Grenoble, France, 2Zhejiang University, Qiushi Academy for Advanced Studies, Hangzhou, China, 3Univ. Grenoble Alpes, CEA, LETI, Clinatec, Grenoble, France, 4Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, Grenoble, France
Patients with locked-in syndrome lose voluntary motor control while cognition is preserved, and speech brain–computer interfaces aim to restore communication by decoding cortical activity. In our case we aim to develop Speech BCIs based on fully implantable epidural electrocorticography (ECoG). Within this pipeline, robust speech detection — knowing when the user is attempting to speak — is a critical front-end step that gates downstream decoding. ECoG decoders, however, drift across sessions and across tasks from one recording block to the next, which forces a brief per-block recalibration — on the order of a few minutes in the clinical regime. In the present study, speech detection was investigated in one tetraplegic patient chronically implanted with the epidural WIMAGINE® implant over the hand area for a motor control clinical trial and able to speak. The goal was to determine whether speech intervals could be identified reliably despite the non-optimal positioning of the electrodes with respect to speech areas in order to anticipate a future dedicated speech BCI paradigm with this type of epidural approach. We evaluate whether enriching pretraining with ECoG-specific perturbations — noising, masking, and mixing across channels and time — yields a backbone that calibrates more reliably on each new block. The task is binary speech detection: every 10 ms, a convolutional network labels the trailing 2-second ECoG window at 600 Hz as speech or silence. Two backbones were pretrained on a 56-minute reference session, with and without the augmentation menu, and calibrated on three later blocks covering reading aloud, sung recall, and free conversation. For each block we crossed pretraining mode (from-scratch vs fine-tuning), augmentation at pretraining (on vs off), and calibration budget (short vs longer, both on the order of minutes), reporting held-out test F1 of the validation-selected seed. We found that fine-tuning the augmentation-pretrained backbone raised free-speech F1 score from 0.380 to 0.643 over from-scratch training under the short budget; on song production blocks, pretraining-time augmentation increased F1 from 0.445 to 0.656, and keeping augmentation on during calibration raised it from 0.656 to 0.764 at matched budget. The best configuration pretrains with augmentation and keeps it active during per-block calibration. Overall, these results demonstrate that self-paced speech intervals can be detected reliably from epidural recordings acquired with 32 electrodes positioned over the hand sensorimotor cortex, despite non-optimal electrode placement for speech decoding. They further highlight the importance of data augmentation and cross-session transfer learning for improving the robustness of CNN-based speech detection in chronic epidural ECoG.
Topic Areas: Speech Motor Control, Speech-Language Treatment