Artificial Intelligence in Medicine: A Comparative Study of ChatGPT and Google Bard in Clinical Diagnostics
Abstract
Introduction
The introduction of Artificial Intelligence (AI) tools like ChatGPT and Google Bard promises transformative advances in clinical diagnostics. The aim of this study is to examine the ability of these two AI tools to diagnose various medical scenarios.
Methods
Experts from varied medical domains curated 20 case scenarios, each paired with its ideal diagnostic answer. Both AI systems, ChatGPT (updated in September 2021) and Google Bard (updated in January 2023), were tasked with diagnosing these cases. Their outcomes were recorded and subsequently assessed by human medical professionals.
Results
In the diagnostic evaluations, ChatGPT achieved an accuracy of 90%, correctly diagnosing 18 out of 20 cases, while Google Bard displayed an 80% accuracy rate, correctly answering 16 questions. Notably, both AIs faltered in specific complex scenarios. For instance, both systems misdiagnosed a labor situation, and while ChatGPT incorrectly identified a case of hypertrophic pyloric stenosis, Google Bard suggested a less suitable diagnostic procedure (pelvic ultrasound) for a 56-year-old patient.
Conclusion
This study showcases the promising capabilities of ChatGPT and Google Bard in the realm of clinical diagnostics, with both AI tools achieving commendable accuracy rates.
References
- Amisha, Malik P, Pathania M, Rathaur VK. Overview of artificial intelligence in medicine. Journal of Family Medicine and Primary Care. 2019;8(7):2328–31. doi:10.4103/jfmpc.jfmpc_440_19
- Salih AM, Mohammed BA, Hasan KM, Fattah FH, Najmadden ZB, Kakamad FH, et al. Mitigating the Burden of meningitis outbreak; ChatGPT and Google Bard Recommendations for the general populations; general practitioners and pediatricians. Barw Medical Journal. 2023;1(2). doi:10.58742/BMJ.V1I2.32
- AYDIN Ö. Google Bard generated literature review: metaverse. Journal of AI. 2023;7(1): 1-14. doi: N/A
- Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT - Reshaping medical education and clinical management. Pakistan Journal of Medical Science. 2023;39(2):605-7. doi:10.12669/pjms.39.2.7653
- Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Medical Education. 2023;9:e45312. doi:10.2196/45312
- Srivastav S, Chandrakar R, Gupta S, Babhulkar V, Agrawal S, Jaiswal A, et al. ChatGPT in Radiology: The Advantages and Limitations of Artificial Intelligence for Medical Imaging Diagnosis. Cureus. 2023;15(7):e41435. doi:10.7759/cureus.41435
- Baumgartner C. The potential impact of ChatGPT in clinical and translational medicine. Clincal and Translational Medicine. 2023;13(3):e1206. doi:10.1002/ctm2.1206
- Suhag A, Kidd J, McGath M, Rajesh R, Gelfinbein J, Cacace N, Monteleone B, Chavez MR. ChatGPT: a pioneering approach to complex prenatal differential diagnosis. American Journal of Obstetrics & Gynecology MFM. 2023;5(8). doi:10.1016/j.ajogmf.2023.101029
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medical. 2019;25(1):44–56. doi:10.1038/s41591-018-0300-7
- Ahmed I, Kajol M, Hasan U, Datta PP, Roy A, Reza MR. ChatGPT vs. Bard: A Comparative Study [Internet]. TechRxiv [Preprint]. 2023. doi:10.36227/techrxiv.23536290.v2
- Jha S, Topol EJ. Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists. JAMA. 2016;316(22):2353-54. doi:10.1001/jama.2016.17438
- Castelvecchi D. Can we open the black box of AI?. Nature News. 2016;538(7623):20-3. doi:10.1038/538020a
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.