Meet OpenAI's doctor-whisperer making ChatGPT better at talking health

If you've noticed ChatGPT giving better advice about your aching back, it's likely thanks to a doctor thousands of miles away.

Speaking 49 languages, these contracted physicians evaluate examples of conversations that users or doctors might have with the chatbot, looking for places where it could confuse its users or put someone in physical danger.

The physician leading these efforts, Rebecca Soskin Hicks, joined OpenAI in 2024 and has since become something of a "doctor-whisperer," working between doctors and the company to improve AI in healthcare.

The stakes can indeed be high. In July, a pastor sued OpenAI over ChatGPT's responses to details about a health crisis with his lungs. He alleged that the GPT-4o model had misdiagnosed and underplayed his symptoms; the lawsuit sought to get OpenAI to pause the operation of healthcare-related products.

Treating chatbots as the whole story behind a medical decision "oversimplifies" the situation, she said, and "risks actually getting in the way of us getting the beneficial impact of this in the hands of as many people as possible."

Add BI in Google so our reporting is easier to find when you’re searching for what matters.

Soskin Hicks said the physicians' network aims to bring in both geographic diversity and a range of specialties.

These doctors work for OpenAI as part-time contractors, from as far away as Kenya, Nepal, and Brazil. As of July, they had reviewed over 700,000 responses from ChatGPT to evaluate how the models could improve. They aren't directly providing data for AI training, though OpenAI and other model developers have increasingly used data from white-collar professionals to improve their tech.

After the doctors evaluate the models, they tell the research team "where the gaps are, where performance could be better," she said. The research team then looks for helpful new data and training methods to help ChatGPT accurately triage emergencies, solicit missing information, communicate uncertainty, and respond with the right level of detail.

Already, OpenAI has made it easy to connect Apple Health and some medical records to ChatGPT, and Singhal touted the latest GPT-6 Astra model as the "state of the art" on a health benchmark test. The company also released a benchmark for testing a model's mental health responses last month.

The physicians' feedback and the models' performance in benchmark tests should be looked at as first steps in evidence, Soskin Hicks said. She hopes that, over time, studies will show that the use of chatbots for health advice actually improves outcomes for both individuals and populations.

Have a tip? Contact this reporter via email at scouncil@businessinsider.com, or over text, Signal, Telegram, or WhatsApp at 415-757-8198. Use a personal email address, a nonwork WiFi network, and a nonwork device; here's our guide to sharing information securely.