Comparative Analysis of ChatGPT and Human Decision-Making in Thyroid and Neck Swellings: A Case-Based Study
Abstract
Introduction
This study aimed to evaluate the performance of Chat Generative Pre-Trained Transformer (ChatGPT), an AI-powered chatbot, in providing treatment recommendations for head and neck nodules.
Methods
Ten diverse cases were examined, including individuals with varying ages and conditions such as thyroid nodules and suspicious neck lesions. The decisions made by ChatGPT were compared to those of physicians. Data were collected from the Smart Health Tower on May 2, 2023.
Results
Analysis of the cases revealed that ChatGPT provided recommendations that aligned with physicians' decisions in seven cases. However, disparities were observed in three cases (N5, N8, N10). Discrepancies were influenced by factors such as nodule size, thyroid function, and the presence of associated symptoms.
Conclusion
The findings suggest that ChatGPT can assist healthcare professionals in formulating individualized diagnoses and treatment strategies for head and neck nodules. However, further research is needed to validate the performance of ChatGPT in larger patient populations and assess its long-term impact on patient outcomes.
References
- Amisha, Malik P, Pathania M, Rathaur VK. Overview of artificial intelligence in medicine. J Fam Med Prim care. 2019 Jul;8(7):2328–31. doi:10.4103/jfmpc.jfmpc_440_19
- Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med Educ. 2023 Feb;9: e45312. doi:10.2196/45312
- Ayers JW, Zhu Z, Poliak A, Leas EC, Dredze M, Hogarth M, et al. Evaluating Artificial Intelligence Responses to Public Health Questions. JAMA Netw Open [Internet]. 2023 Jun 7;6(6):e2317517–e2317517. Available from: doi:10.1001/jamanetworkopen.2023.17517
- ChatGPT sets record for fastest-growing user base-analyst note. Available from: https://www.marketscreener.com/news/latest/ChatGPT-sets-record-for-fastestgrowing-user-base-analyst-note--42873811/.
- ChatGPT — Release Notes | OpenAI Help Center [Internet]. [cited 2023 Jun 25]. Available from: https://help.openai.com/en/articles/6825453-chatgpt-release-notes.
- Dave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations [Internet]. Vol. 6, Frontiers in Artificial Intelligence 2023. doi:10.3389/frai.2023.1169595
- Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT - Reshaping medical education and clinical management. Pakistan J Med Sci. 2023;39(2):605–7. doi:10.12669/pjms.39.2.7653
- Hirosawa T, Harada Y, Yokose M, Sakamoto T, Kawamura R, Shimizu T. Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study. Int J Environ Res Public Health. 2023 Feb;20(4). doi:10.3390/ijerph20043378
- Liao Z, Wang J, Shi Z, Lu L, Tabata H. Revolutionary Potential of ChatGPT in Constructing Intelligent Clinical Decision Support Systems. Annals of biomedical engineering. United States; 2023. doi:10.1007/s10439-023-03288-w
- Lecler A, Duron L, Soyer P. Revolutionizing radiology with GPT-based models: Current applications, future possibilities and limitations of ChatGPT. Diagn Interv Imaging. 2023;104(6):269–74. doi:10.1016/j.diii.2023.02.003
- Balas M, Ing EB. Conversational AI Models for ophthalmic diagnosis: Comparison of ChatGPT and the Isabel Pro Differential Diagnosis Generator. JFO Open Ophthalmol. 2023; 1:100005. doi:10.1016/j.jfop.2023.100005
- Li W, Zhang Y, Chen F. ChatGPT in Colorectal Surgery: A Promising Tool or a Passing Fad? Ann Biomed Eng. 2023; Available from: doi:10.1007/s10439-023-03232-y.
- Ma Y. The potential application of ChatGPT in gastrointestinal pathology. Gastroenterol Endosc. 2023;1(3):130–1. doi:10.1016/j.gande.2023.05.002
- Sinha RK, Deb Roy A, Kumar N, Mondal H. Applicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology. Cureus. 2023; 15(2): e35237. doi:10.7759/cureus.35237
- Giannos P. Evaluating the limits of AI in medical specialisation: ChatGPT’s performance on the UK Neurology Specialty Certificate Examination. BMJ Neurol Open. 2023; 5(1): e000451. doi:10.1136/bmjno-2023-000451
- Baumgartner C. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023; 13(3): e1206. doi:10.1002/ctm2.1206
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56. doi:10.1038/s41591-018-0300-7
- Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017; 2(4):230 LP – 243. doi:10.1136/svn-2017-000101
- Mittelstadt BD, Allo P, Taddeo M, Wachter S, Floridi L. The ethics of algorithms: Mapping the debate. Big Data Soc. 2016; 3(2):2053951716679679. doi:10.1177/2053951716679679
- Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Heal. 2023; 2(2): e0000198. doi:10.1371/journal.pdig.0000198
- Hoch CC, Wollenberg B, Lüers J-C, Knoedler S, Knoedler L, Frank K, et al. ChatGPT’s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch oto-rhino-laryngology Off J Eur Fed Oto-Rhino-Laryngological Soc Affil with Ger Soc Oto-Rhino-Laryngology - Head Neck Surg. 2023. doi:10.1007/s00405-023-08051-4
- Wang B, Wan Z, Li C, Zhang M, Shi Y, Miao X, et al. Identification of benign and malignant thyroid nodules based on dynamic AI ultrasound intelligent auxiliary diagnosis system. Front Endocrinol (Lausanne). 2022; 13:1018321. doi:10.3389/fendo.2022.1018321
- Nguyen DT, TD, Batchuluun G, Yoon HS, Park KR. Artificial Intelligence-Based Thyroid Nodule Classification Pham Using Information from Spatial and Frequency Domains. J Clin Med. 2019; 8(11). doi:10.3390/jcm8111976
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.