Dr. Adam Rodman, an experienced doctor at Beth Israel Deaconess Medical Center in Boston, believed that chatbots powered by artificial intelligence (AI) would be a huge help for doctors in diagnosing diseases. He thought AI could give doctors a boost in their work, making them more accurate and faster. But after testing it, he was stunned by the results.

In a study Dr. Rodman helped design, doctors who had access to ChatGPT-4, a popular AI chatbot, only performed slightly better than doc...