Automatic classification of herbal formula documents related to diseases using LLM-based embedding vectors

LLM 기반 임베딩 벡터를 활용한 질병과 관련된 전통 한약 처방 문서의 자동 분류

Objectives: This study focuses on the development of an automatic classification system for herbal formula documents related to diseases. Methods: We used 740 abstracts of herbal formula documents associated with diseases as positive samples, and sampled negative documents from the entire PubMed database. For feature extraction, we utilized text-embedding-ada-002 and OpenAI's bert-base-uncased embedding vectors based on BERT, and employed XGBoost and SVM as classification models. The performance of the classification models was evaluated using Precision, Recall, F1-Score, AUC-ROC, and AUC-PR metrics. Results: SVM consistently outperformed XGBoost, and the classification performance differed depending on whether the negative documents were sampled from the entire PubMed database or from traditional medicine literature. Furthermore, as the proportion of negative documents in the training data increased, the model's classification performance decreased. A comparison between PubMed's default ranking and the ranking generated by our automatic classification system showed that our method ranked disease-related prescription documents higher. Conclusions: Our model demonstrates that our approach can improve the efficiency of searching and collecting disease-related herbal formula literature.