• Title/Summary/Keyword: Mean Reciprocal Rank

Search Result 16, Processing Time 0.021 seconds

Using Semantic Knowledge in the Uyghur-Chinese Person Name Transliteration

  • Murat, Alim;Osman, Turghun;Yang, Yating;Zhou, Xi;Wang, Lei;Li, Xiao
    • Journal of Information Processing Systems
    • /
    • v.13 no.4
    • /
    • pp.716-730
    • /
    • 2017
  • In this paper, we propose a transliteration approach based on semantic information (i.e., language origin and gender) which are automatically learnt from the person name, aiming to transliterate the person name of Uyghur into Chinese. The proposed approach integrates semantic scores (i.e., performance on language origin and gender detection) with general transliteration model and generates the semantic knowledge-based model which can produce the best candidate transliteration results. In the experiment, we use the datasets which contain the person names of different language origins: Uyghur and Chinese. The results show that the proposed semantic transliteration model substantially outperforms the general transliteration model and greatly improves the mean reciprocal rank (MRR) performance on two datasets, as well as aids in developing more efficient transliteration for named entities.

Improving Accuracy of Chapter-level Lecture Video Recommendation System using Keyword Cluster-based Graph Neural Networks

  • Purevsuren Chimeddorj;Doohyun Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.7
    • /
    • pp.89-98
    • /
    • 2024
  • In this paper, we propose a system for recommending lecture videos at the chapter level, addressing the balance between accuracy and processing speed in chapter-level video recommendations. Specifically, it has been observed that enhancing recommendation accuracy reduces processing speed, while increasing processing speed decreases accuracy. To mitigate this trade-off, a hybrid approach is proposed, utilizing techniques such as TF-IDF, k-means++ clustering, and Graph Neural Networks (GNN). The approach involves pre-constructing clusters based on chapter similarity to reduce computational load during recommendations, thereby improving processing speed, and applying GNN to the graph of clusters as nodes to enhance recommendation accuracy. Experimental results indicate that the use of GNN resulted in an approximate 19.7% increase in recommendation accuracy, as measured by the Mean Reciprocal Rank (MRR) metric, and an approximate 27.7% increase in precision defined by similarities. These findings are expected to contribute to the development of a learning system that recommends more suitable video chapters in response to learners' queries.

Open-domain Question Answering Using Lexico-Semantic Patterns (Lexico-Semantic Pattern을 이용한 오픈 도메인 질의 응답 시스템)

  • Lee, Seung-Woo;Jung, Han-Min;Kwak, Byung-Kwan;Kim, Dong-Seok;Cha, Jeong-Won;An, Joo-Hui;Lee, Gary Geun-Bae;Kim, Hark-Soo;Kim, Kyung-Sun;Seo, Jung-Yun
    • Annual Conference on Human and Language Technology
    • /
    • 2001.10d
    • /
    • pp.538-545
    • /
    • 2001
  • 본 연구에서는 오픈 도메인에서 동작할 수 있는 질의 응답 시스템(Open-domain Question Answer ing System)을 구현하고 영어권 TREC에 참가한 결과를 기술하였다. 정답 유형을 18개의 상위 노드를 갖는 계층구조로 분류하였고, 질문 처리에서는 LSP(Lexico-Semantic Pattern)으로 표현된 문법을 사용하여 질문의 정답 유형을 결정하고, lemma 형태와 WordNet 의미, stem 형태의 3가지 유형의 키워드로 구성된 질의를 생성한다. 이 질의를 바탕으로, 패시지 선택에서는 문서검색 엔진에 의해 검색된 문서들을 문장단위로 나눠 정수를 계산하고, 어휘체인(Lexical Chain)을 고려하여 인접한 문장을 결합하여 패시지를 구성하고 순위를 결정한다. 상위 랭크의 패시지를 대상으로, 정답 처리에서는 질문의 정답 유형에 따라 품사와 어휘, 의미 정보로 기술된 LSP 매칭과 AAO (Abbreviation-Appositive-Definition) 처리를 통해 정답을 추출하고 정수를 계산하여 순위를 결정한다. 구현된 시스템의 성능을 평가하기 위해 TREC10 QA Track의 main task의 질문들 중, 200개의 질문에 대해 TRIC 방식으로 자체 평가를 한 결과, MRR(Mean Reciprocal Rank)은 0.341로 TREC9의 상위 시스템들과 견줄 만한 성능을 보였다.

  • PDF

A New Similarity Measure for Improving Ranking in QA Systems (질의응답시스템 응답순위 개선을 위한 새로운 유사도 계산방법)

  • Kim Myung-Gwan;Park Young-Tack
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.6
    • /
    • pp.529-536
    • /
    • 2004
  • The main idea of this paper is to combine position information in sentence and query type classification to make the documents ranking to query more accessible. First, the use of conceptual graphs for the representation of document contents In information retrieval is discussed. The method is based on well-known strategies of text comparison, such as Dice Coefficient, with position-based weighted term. Second, we introduce a method for learning query type classification that improves the ability to retrieve answers to questions from Question Answering system. Proposed methods employ naive bayes classification in machine learning fields. And, we used a collection of approximately 30,000 question-answer pairs for training, obtained from Frequently Asked Question(FAQ) files on various subjects. The evaluation on a set of queries from international TREC-9 question answering track shows that the method with machine learning outperforms the underline other systems in TREC-9 (0.29 for mean reciprocal rank and 55.1% for precision).

Detection of Similar Answers to Avoid Duplicate Question in Retrieval-based Automatic Question Generation (검색 기반의 질문생성에서 중복 방지를 위한 유사 응답 검출)

  • Choi, Yong-Seok;Lee, Kong Joo
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.8 no.1
    • /
    • pp.27-36
    • /
    • 2019
  • In this paper, we propose a method to find the most similar answer to the user's response from the question-answer database in order to avoid generating a redundant question in retrieval-based automatic question generation system. As a question of the most similar answer to user's response may already be known to the user, the question should be removed from a set of question candidates. A similarity detector calculates a similarity between two answers by utilizing the same words, paraphrases, and sentential meanings. Paraphrases can be acquired by building a phrase table used in a statistical machine translation. A sentential meaning's similarity of two answers is calculated by an attention-based convolutional neural network. We evaluate the accuracy of the similarity detector on an evaluation set with 100 answers, and can get the 71% Mean Reciprocal Rank (MRR) score.

Construction of Pilot System to Improve Search Quality in National Archives of Korea Portal and Effects Validation (국가기록포털 검색 품질 개선을 위한 파일럿 시스템 구축 및 실효성 검증)

  • Hyeon-Gi So;Gyung Rok Yeom;Hyo-Jung Oh
    • Journal of Korean Society of Archives and Records Management
    • /
    • v.23 no.2
    • /
    • pp.117-135
    • /
    • 2023
  • The National Archives of Korea (NAK) operates the NAK Portal as a record search system. However, user search satisfaction is too low, and the number of visitors to the portal is gradually decreasing. This study identifies the portal's issues, proposes feasible improvements, and constructs a pilot system to validate the solutions. The preliminary assessment revealed six major issues, such as poor search tool performance and the lack of consistency in search results. After clarifying the improvement measures, a pilot system was established and compared with the National Records Portal. The evaluation showed significant performance improvements in the pilot system, such as Precision, Recall, and Mean Reciprocal Rank (MRR).