CoLive Voice: diagnosing serious illnesses using the voice

The human voice is a powerful communication tool that conveys feelings and emotions, but can also point to changes in a person's health. In conjunction with the intensive use of digital tools and artificial intelligence in healthcare, this provides a strong argument for the use of speech-based technologies for diagnostic and medical purposes.

Support with virtual medicine and remote monitoring of patients 

Vocal biomarkers could soon be used in a variety of contexts such as tele- or virtual medicine, remotely monitoring patients between examinations, or evaluating the effectiveness of a drug in clinical trials. For this purpose, voice recordings, clinical and epidemiological data, as well as treatment results, should be recorded simultaneously and systematically using well-structured methods.

The Deep Digital Phenotyping (DDP) research unit at the Luxembourg Institute of Health (LIH) recently launched the “CoLive Voice” study, which aims to collect tens of thousands of volunteer voice samples. It will then analyse them to identify vocal biomarkers for a variety of chronic conditions and symptoms, ranging from stress, anxiety and depression to cancer, diabetes, multiple sclerosis, inflammatory bowel disease and even COVID-19.

Extracting significant audio features from voice samples 

Dr. Guy Fagherazzi, Director of DoPH and Study Director of CoLive Voice, explains:

"A vocal biomarker is a feature or combination of features of the voice that can be associated with a particular clinical outcome and is therefore a valuable tool for monitoring patients, making a clinical diagnosis, assessing the severity of a disease and even assisting in the development of new drugs."

Participants in the study must answer a detailed questionnaire on their health status, using the CoLive Voice web application. The questionnaire, which remains anonymous, includes clinical and epidemiological aspects such as lifestyle factors, personal information on symptoms, ongoing treatments and diagnosed diseases. Participants must then submit five different voice samples, during which they read short texts, cough, breathe deeply and count to 20, among other things. Researchers at the LIH will then process the recorded data and extract the most significant audio features depending on the voice disorder, illness or type of voice sample.

Analysis of acoustic and linguistic features 

"Acoustic features extracted from the recordings of a drawn out vowel such as "aaaa"can help us identify Parkinson's, for example, while linguistic features extracted from spontaneous or semi-spontaneous speech might be better suited to mental disorders," explains Aurélie Fischer, the CoLive Voice project coordinator.

With audio characteristics such as this, models for machine learning and deep learning can then be coded in order to automatically predict or classify relevant clinical, medical or epidemiological outcomes of any kind, either with purely vocal characteristics or in combination with other health data.

Developing a multilingual audio database to identify vocal biomarkers 

People aged 15 and over, from all over the world, can take part in the CoLive Voice project, regardless of their state of health. In addition to people without any particular health problems, researchers are looking for patients living with cancer, diabetes, multiple sclerosis, inflammatory bowel disease, or COVID-19.

"Although it is a Luxembourg-based initiative, the strength of the study lies in its international dimension. We are hoping more than 50,000 people worldwide will take part and thus contribute to making CoLive Voice an international and multilingual audio database for the identification of vocal biomarkers," says Dr. Fagherazzi.

Available in four languages 

Participation in the CoLive Voice project is voluntary, on a one-time basis, and will remain completely anonymous. Additional information about the science behind it will be presented to participants in the form of "fun facts" about the voice. The app is available in English, French, German and Spanish, and can be found at the following link: www.colivevoice.org

Author: LIH

Editor: Uwe Hentschel