A recent study conducted by researchers from Harvard Medical School and Stanford University revealed that the artificial intelligence model “ChatGPT o1” (in experimental mode “Preview-01”) from the company “OpenAI” may outperform doctors in diagnosing… Complex medical cases.
During the study, the model underwent a comprehensive series of medical diagnostic tests, and the results showed that it achieved qualitative leaps compared to previous versions. The “o1-preview” model was able to accurately diagnose 78.3% of the cases analyzed.
In a direct comparison that included 70 specific cases, the system’s accuracy rose to 88.6%, far superior to the previous GPT-4 system, which recorded a rate of 72.9%. The system proved its high efficiency in the field of medical reasoning, as it obtained high scores in 78 out of 80 cases according to the R-IDEA scale used to evaluate the quality of reasoning.
The researchers indicated that the training data for the model may include some of the cases used in the study, but the model’s performance remained high when tested on new cases that it had not previously dealt with, with a slight decrease in performance.
The researchers explained that the model’s detailed answers contributed to raising his evaluation, stressing that the study was limited to his performance alone without studying how he cooperated with doctors.
The Preview-01 model shows superiority in critical thinking tasks, such as diagnosis and making treatment recommendations, but faces difficulties in abstract tasks, such as estimating probabilities.
OpenAI recently announced the launch of the final full version, O1, in addition to the new version, O3, which demonstrated significant improvements in analytical thinking.