October 6, 2026
Can an AI system listen to a patient describe their symptoms and determine how urgently they may need medical attention?
My interest in that question began while volunteering at Grady Hospital in Atlanta. I saw how difficult it could sometimes be for patients to communicate what they were experiencing, particularly when English was not their first language. Conversations could slow down or require repetition, not because anyone stopped trying to understand, but because important details could be difficult to communicate across a language barrier.
That experience first led me to study cultural and linguistic differences in AI mental-health chatbots. Presenting that research at the Grady Research Symposium pushed me to think about what could come next: rather than only measuring where existing AI systems struggled, could I build something specifically around the moment when a patient first describes what is wrong? That question became OneVoice Triage, a voice-first, multilingual emergency assessment system designed to turn natural spoken descriptions into structured clinical predictions.
I wanted OneVoice to make initial triage more accessible and inclusive, helping patients explain their symptoms in their preferred language and receive attention that matches their urgency. For doctors and nurses, it could provide a rapid, data-informed assessment of urgency and possible diagnoses, helping them recognize urgent cases sooner and prioritize patients more consistently.
A patient begins by describing symptoms aloud rather than filling out a conventional questionnaire. OneVoice processes the speech, identifies the language, and converts the patient’s description into a standardized representation for analysis. The system then produces two main outputs: an urgency classification (LOW, MEDIUM, HIGH, or CRITICAL) and a ranked list of possible diagnoses.
I designed and developed OneVoice’s processing pipeline using Python, Java, and Kotlin. I prepared and processed the medical data, fine-tuned the Bio_ClinicalBERT-based classifier using DDXPlus and MIMIC-IV-ED, and programmed the Android app’s audio input, language selection, symptom analysis, and results display. I connected speech recognition, language identification, multilingual translation, and medical classification into a working system, then debugged and tested how information moved through each stage. A substantial part of my work involved balancing model performance, response time, and memory use so the complete pipeline could run directly on a phone.
I built OneVoice to support spoken symptom descriptions in nearly 100 languages, with accessibility central to its design. Once installed and set up, it works without an internet connection. Speech recognition, translation, and medical classification run directly on the device, and the patient’s audio stays on the phone. This makes offline operation particularly relevant to rural communities, clinics with limited resources, and disaster areas with unreliable internet or no connectivity.
Testing examined both predictive performance and response time. The system achieved a macro-F1 of 0.74 for urgency classification, with per-class accuracy ranging from 70% to 92%, and the correct urgency level appeared among the top three predictions in 89% of evaluated cases. End-to-end processing latency was approximately 2.2 to 2.9 seconds in testing.
Working on OneVoice also changed how I thought about medical AI. Building a system is only part of the challenge; it also has to be tested against the different ways real people communicate and against the populations it is intended to serve. A model that performs well overall can still fail in important ways when language, context, or patient populations change. To test OneVoice beyond the data, I asked emergency physicians at Grady Hospital and Emory to review the app, and their feedback shaped how I think about where it could fit into real triage workflows.
The contribution of the project is broader than simply predicting a diagnosis. OneVoice investigates whether speech recognition, multilingual language processing, and clinical machine learning can be integrated into one lightweight triage pipeline that assesses both what might be wrong and how urgently the patient may need care.
Ultimately, the project grew from something I observed at Grady into a question I could actually investigate: Can technology preserve more of what a patient is trying to communicate at the very beginning of care? OneVoice is one attempt to explore that question. It is intended as an emergency-assessment research system rather than a replacement for professional diagnosis or clinical judgment.
Click here to download the Logbook
Click here to download the paper
Thanks for reading my blog !
MedMuse