Korean Journal of Radiology

  • 제25권2호
  • /
  • Pages.126-133
  • /
  • 2024
  • /
  • 1229-6929(pISSN)
  • /
  • 2005-8330(eISSN)
대한영상의학회 (The Korean Society of Radiology)
권호 표지
DOI QR코드

DOI QR Code

Large Language Models: A Guide for Radiologists

  • Sunkyu Kim (Department of Computer Science and Engineering, Korea University) ;
  • Choong-kun Lee (Division of Medical Oncology, Department of Internal Medicine, Yonsei University College of Medicine) ;
  • Seung-seob Kim (Department of Radiology and Research Institute of Radiological Science, Severance Hospital, Yonsei University College of Medicine)
  • 투고 : 2023.10.12
  • 심사 : 2023.12.18
  • 발행 : 2024.02.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Large language models (LLMs) have revolutionized the global landscape of technology beyond natural language processing. Owing to their extensive pre-training on vast datasets, contemporary LLMs can handle tasks ranging from general functionalities to domain-specific areas, such as radiology, without additional fine-tuning. General-purpose chatbots based on LLMs can optimize the efficiency of radiologists in terms of their professional work and research endeavors. Importantly, these LLMs are on a trajectory of rapid evolution, wherein challenges such as "hallucination," high training cost, and efficiency issues are addressed, along with the inclusion of multimodal inputs. In this review, we aim to offer conceptual knowledge and actionable guidance to radiologists interested in utilizing LLMs through a succinct overview of the topic and a summary of radiology-specific aspects, from the beginning to potential future directions.

키워드

참고문헌

  1. Harris ZS. Distributional structure. Word 1954;10:146-162
  2. Le Q, Mikolov T. Distributed representations of sentences and documents [accessed on August 18, 2023]. Available at: https://proceedings.mlr.press/v32/le14.html?ref=https://githubhelp.com
  3. Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1986;323:533-536
  4. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9:1735-1780
  5. Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv [Preprint]. 2014 [accessed on August 18, 2023]. Available at: https://doi.org/10.48550/arXiv.1406.1078
  6. Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks [accessed on August 18, 2023]. Available at: https://proceedings.neurips.cc/paper/2014/hash/a14ac55a4f27472c5d894ec1c3c743d2-Abstract.html
  7. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need [accessed on August 18, 2023]. Available at: https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  8. Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv [Preprint]. 2018 [accessed on August 18, 2023]. Available at: https://doi.org/10.48550/arXiv.1810.04805
  9. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al. Language models are few-shot learners [accessed on August 18, 2023]. Available at: https://proceedings.neurips.cc/paper_files/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html?utm_medium=email&utm_source=transaction
  10. Jung KH. Uncover this tech term: foundation model. Korean J Radiol 2023;24:1038-1041
  11. Hwang SI, Lim JS, Lee RW, Matsui Y, Iguchi T, Hiraki T, et al. Is ChatGPT a "fire of prometheus" for non-native English-speaking researchers in academic writing? Korean J Radiol 2023;24:952-959
  12. Koga S. The integration of large language models such as ChatGPT in scientific writing: harnessing potential and addressing pitfalls. Korean J Radiol 2023;24:924-925
  13. Park SH. Use of generative artificial intelligence, including large language models such as ChatGPT, in scientific publications: policies of KJR and prominent authorities. Korean J Radiol 2023;24:715-718
  14. Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA 2023;329:842-844
  15. Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT. Radiology 2023;307:e230424
  16. Rahsepar AA, Tavakoli N, Kim GHJ, Hassani C, Abtin F, Bedayat A. How AI responds to common lung cancer questions: ChatGPT vs Google Bard. Radiology 2023;307:e230922
  17. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepano C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health 2023;2:e0000198
  18. Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations. Radiology 2023;307:e230582
  19. Bhayana R, Bleakney RR, Krishna S. GPT-4 in radiology: improvements in advanced reasoning. Radiology 2023;307:e230987
  20. Ueda D, Mitsuyama Y, Takita H, Horiuchi D, Walston SL, Tatekawa H, et al. ChatGPT's diagnostic performance from patient history and imaging findings on the diagnosis please quizzes. Radiology 2023;308:e231040
  21. Kottlors J, Bratke G, Rauen P, Kabbasch C, Persigehl T, Schlamann M, et al. Feasibility of differential diagnosis based on imaging patterns using a large language model. Radiology 2023;308:e231167
  22. Sun Z, Ong H, Kennedy P, Tang L, Chen S, Elias J, et al. Evaluating GPT4 on impressions generation in radiology reports. Radiology 2023;307:e231259
  23. Adams LC, Truhn D, Busch F, Kader A, Niehues SM, Makowski MR, et al. Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting: a multilingual feasibility study. Radiology 2023;307:e230725
  24. Fink MA, Bischoff A, Fink CA, Moll M, Kroschke J, Dulz L, et al. Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer. Radiology 2023;308:e231362
  25. Lyu Q, Tan J, Zapadka ME, Ponnatapura J, Niu C, Myers KJ, et al. Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: promising results, limitations, and potential. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2303.09038
  26. Doshi R, Amin K, Khosla P, Bajaj S, Chheang S, Forman HP. Utilizing large language models to simplify radiology reports: a comparative analysis of ChatGPT3.5, ChatGPT4.0, Google Bard, and Microsoft Bing. medRxiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.1101/2023.06.04.23290786
  27. Rau A, Rau S, Zoeller D, Fink A, Tran H, Wilpert C, et al. A context-based chatbot surpasses trained radiologists and generic ChatGPT in following the ACR appropriateness guidelines. Radiology 2023;308:e230970
  28. Gertz RJ, Bunck AC, Lennartz S, Dratsch T, Iuga AI, Maintz D, et al. GPT-4 for automated determination of radiological study and protocol based on radiology request forms: a feasibility study. Radiology 2023;307:e230877
  29. Rao A, Kim J, Kamineni M, Pang M, Lie W, Succi MD. Evaluating ChatGPT as an adjunct for radiologic decision-making. medRxiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.1101/2023.02.02.23285399
  30. Wu Z, Zhang L, Cao C, Yu X, Dai H, Ma C, et al. Exploring the trade-offs: unified large language models vs local fine-tuned models for highly-specific radiology NLI task. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2304.09138
  31. Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature 2023;620:172-180
  32. Singhal K, Tu T, Gottweis J, Sayres R, Wulczyn E, Hou L, et al. Towards expert-level medical question answering with large language models. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2305.09617
  33. Wang G, Yang G, Du Z, Fan L, Li X. ClinicalGPT: large language models finetuned with diverse medical data and comprehensive evaluation. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2306.09968
  34. Liu Z, Zhong A, Li Y, Yang L, Ju C, Wu Z, et al. Radiology-GPT: a large language model for radiology. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2306.08666
  35. Li H, Zhu J, Jiang X, Zhu X, Li H, Yuan C, et al. Uni-perceiver v2: a generalist model for large-scale vision and vision-language tasks [accessed on October 2, 2023]. Available at: https://openaccess.thecvf.com/content/CVPR2023/html/Li_Uni-Perceiver_v2_A_Generalist_Model_for_Large-Scale_Vision_and_Vision-Language_CVPR_2023_paper.html
  36. Zhang K, Yu J, Yan Z, Liu Y, Adhikarla E, Fu S, et al. BiomedGPT: a unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2305.17100
  37. Elkhatat AM. Evaluating the authenticity of ChatGPT responses: a study on text-matching capabilities. Int J Educ Integr 2023;19:15
  38. Bender EM, Gebru T, McMillan-Major A, Shmitchell S. On the dangers of stochastic parrots: can language models be too big? [accessed on October 2, 2023]. Available at: https://dl.acm.org/doi/abs/10.1145/3442188.3445922
  39. Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks [accessed on October 2, 2023]. Available at: https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
  40. Mukherjee P, Hou B, Lanfredi RB, Summers RM. Feasibility of using the privacy-preserving large language model Vicuna for labeling radiology reports. Radiology 2023;309:e231147
  41. Driess D, Xia F, Sajjadi MSM, Lynch C, Chowdhery A, Ichter B, et al. PaLM-E: an embodied multimodal language model. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2303.03378
  42. OpenAI. ChatGPT can now see, hear, and speak [accessed on October 2, 2023]. Available at: https://openai.com/blog/chatgpt-can-now-see-hear-and-speak
  43. Tu T, Azizi S, Driess D, Schaekermann M, Amin M, Chang PC, et al. Towards generalist biomedical AI. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2307.14334
  44. Wu C, Zhang X, Zhang Y, Wang Y, Xie W. Towards generalist foundation model for radiology by leveraging web-scale 2D&3D medical data. arXiv [Preprint]. 2023 [accessed on October 2, 2023]. Available at: https://doi.org/10.48550/arXiv.2308.02463
  45. Delbrouck JB, Varma M, Chambon P, Langlotz C. Overview of the RadSum23 shared task on multi-modal and multi-anatomical radiology report summarization [accessed on October 2, 2023]. Available at: https://aclanthology.org/2023.bionlp-1.45/
  46. Fei N, Lu Z, Gao Y, Yang G, Huo Y, Wen J, et al. Towards artificial general intelligence via a multimodal foundation model. Nat Commun 2022;13:3094