Archives of Craniofacial Surgery (대한두개안면성형외과학회지)

Korean Cleft Palate-Craniofacial Association (대한두개안면성형외과학회)
권호 표지
DOI QR코드

DOI QR Code

The new frontier: utilizing ChatGPT to expand craniofacial research

  • Andi Zhang (Division of Plastic and Reconstructive Surgery, Saint Louis University School of Medicine) ;
  • Ethan Dimock (Oakland University William Beaumont School of Medicine) ;
  • Rohun Gupta (Division of Plastic and Reconstructive Surgery, Saint Louis University School of Medicine) ;
  • Kevin Chen (Division of Plastic and Reconstructive Surgery, Saint Louis University School of Medicine)
  • Received : 2024.02.29
  • Accepted : 2024.06.11
  • Published : 2024.06.20
Download PDF
⟨ Previous Next ⟩

Abstract

Background: Due to the importance of evidence-based research in plastic surgery, the authors of this study aimed to assess the accuracy of ChatGPT in generating novel systematic review ideas within the field of craniofacial surgery. Methods: ChatGPT was prompted to generate 20 novel systematic review ideas for 10 different subcategories within the field of craniofacial surgery. For each topic, the chatbot was told to give 10 "general" and 10 "specific" ideas that were related to the concept. In order to determine the accuracy of ChatGPT, a literature review was conducted using PubMed, CINAHL, Embase, and Cochrane. Results: In total, 200 total systematic review research ideas were generated by ChatGPT. We found that the algorithm had an overall 57.5% accuracy at identifying novel systematic review ideas. ChatGPT was found to be 39% accurate for general topics and 76% accurate for specific topics. Conclusion: Craniofacial surgeons should use ChatGPT as a tool. We found that ChatGPT provided more precise answers with specific research questions than with general questions and helped narrow down the search scope, leading to a more relevant and accurate response. Beyond research purposes, ChatGPT can augment patient consultations, improve healthcare equity, and assist in clinical decision-making. With rapid advancements in artificial intelligence (AI), it is important for plastic surgeons to consider using AI in their clinical practice to improve patient-centered outcomes.

Keywords

References

  1. Sejnowski TJ. Large language models and the reverse Turing test. Neural Comput 2023;35:309-42. 
  2. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepano C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health 2023;2:e0000198. 
  3. Teebagy S, Colwell L, Wood E, Yaghy A, Faustina M. Improved performance of ChatGPT-4 on the OKAP examination: a comparative study with ChatGPT-3.5. J Acad Ophthalmol (2017) 2023;15:e184-7. 
  4. Humar P, Asaad M, Bengur FB, Nguyen V. ChatGPT is equivalent to first-year plastic surgery residents: evaluation of ChatGPT on the plastic surgery in-service examination. Aesthet Surg J 2023;43:NP1085-9. 
  5. Kang E, Gillespie BM, Tobiano G, Chaboyer W. Discharge education delivered to general surgical patients in their management of recovery post discharge: a systematic mixed studies review. Int J Nurs Stud 2018;87:1-13. 
  6. Kumar B. GPT-1, GPT-2 and GPT-3 models explained: learn the evolution of AI language models [Internet]. 360DigiTMG; c2023 [cited 2023 Sep 29]. https://360digitmg.com/blog/types-of-gpt-in-artificial-intelligence 
  7. Osmanovic-Thunstrom A, Steingrimsson S, Thunstrom AO. Can GPT-3 write an academic paper on itself, with minimal human input? Archive ouverte HAL [Preprint] 2022 Jun 21 [cite 2023 Sep 29]. https://hal.science/hal-03701250 
  8. Hansdorfer MA, Horen SR, Alba BE, Akin JN, Dorafshar AH, Becerra AZ. The 100 most-disruptive articles in plastic and reconstructive surgery and sub-specialties (1954-2014). Plast Reconstr Surg Glob Open 2021;9:e3446. 
  9. Grunwald T, Krummel T, Sherman R. Advanced technologies in plastic surgery: how new innovations can improve our training and practice. Plast Reconstr Surg 2004;114:1556-67. 
  10. Gupta R, Park JB, Bisht C, Herzog I, Weisberger J, Chao J, et al. Expanding cosmetic plastic surgery research with ChatGPT. Aesthet Surg J 2023;43:930-7. 
  11. Gupta R, Herzog I, Park JB, Weisberger J, Firouzbakht P, Ocon V, et al. Performance of ChatGPT on the plastic surgery inservice training examination. Aesthet Surg J 2023;43:NP1078-82. 
  12. Gan C, Mori T. Sensitivity and robustness of large language models to prompt template in Japanese text classification tasks. arXiv [Preprint] 2023 Jun 8 [cited 2023 Sep 29]. https://doi.org/10.48550/arXiv.2305.08714 
  13. Dash D, Thapa R, Banda JM, Swaminathan A, Cheatham M, Kashyap M, et al. Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery. arXiv [Preprint] 2023 May 1 [cited 2023 Sep 29]. https://doi.org/10.48550/arXiv.2304.13714 
  14. Bhatti BM. The art and science of crafting effective prompts for LLMs [Internet]. Medium; c2023 [cited 2023 Sep 29]. https://thebabar.medium.com/the-art-and-science-of-crafting-effective-prompts-for-llms-e04447e8f96a 
  15. Liu S, Wright AP, Patterson BL, Wanderer JP, Turer RW, Nelson SD, et al. Using AI-generated suggestions from ChatGPT to optimize clinical decision support. J Am Med Inform Assoc 2023;30:1237-45. 
  16. Barr K. GPT-4 is a giant black box and its training data remains a mystery [Internet]. Gizmodo; c2023 [cited 2023 Sep 29]. https://gizmodo.com/chatbot-gpt4-open-ai-ai-bing-microsoft-1850229989 
  17. Gupta R, Herzog I, Najafali D, Firouzbakht P, Weisberger J, Mailey BA. Application of GPT-4 in cosmetic plastic surgery: does updated mean better? Aesthet Surg J 2023;43:NP666-9. 
  18. Macdonald C, Adeloye D, Sheikh A, Rudan I. Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. J Glob Health 2023;13:01003. 
  19. Seth I, Cox A, Xie Y, Bulloch G, Hunter-Smith DJ, Rozen WM, et al. Evaluating chatbot efficacy for answering frequently asked questions in plastic surgery: a ChatGPT case study focused on breast augmentation. Aesthet Surg J 2023;43:1126-35.