What evidence do we need to use any of this? The bar is higher for diagnostic programs than for programs that write our notes. But the way we usually test advances in medicine — a rigorously designed randomized clinical trial that takes years — won’t work here. After all, when the test was complete, the technology would have changed. Furthermore, the reality is that these technologies will find their way into our daily practice whether they are tested or not.
Dr. Adam Rodman, an internist at Beth Israel Deaconess Hospital in Boston and a historian, found that most of his medical students already use Chat GPT, to help them during rounds or even to help predict test questions. Curious about how AI would perform on difficult medical cases, Dr. Rodman issued the infamous challenge New England Journal of Medicine weekly case – and found that the program suggested the correct diagnosis in a list of possible diagnoses just over 60 percent of the time. This performance is most likely better than any individual could perform.
How those skills translate to the real world remains to be seen. But even as he prepares to embrace new technology, Dr. Rodman wonders if something will be lost. After all, the training of doctors has long followed a clear process – we see patients, we struggle with their care in a controlled environment and we do it again until we finish our training. But with AI, there is the real possibility that doctors in training could rely on these programs to do the hard work of generating a diagnosis, rather than learning to do it themselves. If you never sorted through the mess of seemingly unrelated symptoms to arrive at a possible diagnosis, but instead relied on a computer, how do you learn the thought processes necessary for excellence as a doctor?
“In the very near future, we’re looking at a time where the next generation coming up won’t develop these skills the same way we did,” said Dr. Rodman. Even when it comes to AI writing our notes for us, Dr. Rodman sees a trade-off. After all, grades are not simply a chore; they also represent a time to take stock, to review the data and consider what’s next for our patients. If we offload that work, we certainly gain time, but maybe we also lose something.
But there is a balance here. Perhaps the diagnostics offered by AI will become an adjunct to our own thought processes, not replacing us but allowing all of our tools to improve. Especially for those who work in settings with limited specialists for consultation, AI could bring everyone to the same standard. At the same time, patients will use these technologies, ask questions and come to us with possible answers. This democratization of information is already happening and will only increase.
Perhaps being an expert does not mean being a source of information but synthesizing and communicating and using judgment to make difficult decisions. AI may be part of that process, just one more tool we use, but it will never replace a hand at the bedside, eye contact, understanding – what it is to be a doctor.
A few weeks ago, I downloaded the Chat GPT app. I asked it all kinds of questions, from the medical to the personal. And when I’m working in the intensive care unit next, when I’m faced with a question during rounds, I might open the app and see what AI has to say.