ChatGPT has outperformed trainee doctors in evaluating complex respiratory cases, including cystic fibrosis, asthma, and chest infections, according to a study presented at the European Respiratory Society (ERS) Congress in Vienna.
The research, led by Dr. Manjith Narayanan of the Royal Hospital for Children and Young People in Edinburgh, highlighted that Google’s Bard also exceeded trainee performance in certain areas, while Microsoft’s Bing matched trainees’ abilities.
Dr. Narayanan explained, “Large language models, like ChatGPT, have come into prominence in the last year and a half with their ability to seemingly understand natural language and provide responses that can adequately simulate a human-like conversation. These tools have several potential applications in medicine. My motivation to carry out this research was to assess how well LLMs are able to assist clinicians in real life.”
The study used clinical scenarios frequently encountered in paediatric respiratory medicine, evaluated by six experts. Trainee doctors, with less than four months of paediatric experience, and the three chatbots were assessed for their diagnostic responses. ChatGPT’s solutions scored highest among the tools tested.
