• Title/Summary/Keyword: Corpus construction

Search Result 112, Processing Time 0.027 seconds

On the Analysis of Natural Language Processing Morphology for the Specialized Corpus in the Railway Domain

  • Won, Jong Un;Jeon, Hong Kyu;Kim, Min Joong;Kim, Beak Hyun;Kim, Young Min
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.4
    • /
    • pp.189-197
    • /
    • 2022
  • Today, we are exposed to various text-based media such as newspapers, Internet articles, and SNS, and the amount of text data we encounter has increased exponentially due to the recent availability of Internet access using mobile devices such as smartphones. Collecting useful information from a lot of text information is called text analysis, and in order to extract information, it is performed using technologies such as Natural Language Processing (NLP) for processing natural language with the recent development of artificial intelligence. For this purpose, a morpheme analyzer based on everyday language has been disclosed and is being used. Pre-learning language models, which can acquire natural language knowledge through unsupervised learning based on large numbers of corpus, are a very common factor in natural language processing recently, but conventional morpheme analysts are limited in their use in specialized fields. In this paper, as a preliminary work to develop a natural language analysis language model specialized in the railway field, the procedure for construction a corpus specialized in the railway field is presented.

A study on performance improvement considering the balance between corpus in Neural Machine Translation (인공신경망 기계번역에서 말뭉치 간의 균형성을 고려한 성능 향상 연구)

  • Park, Chanjun;Park, Kinam;Moon, Hyeonseok;Eo, Sugyeong;Lim, Heuiseok
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.5
    • /
    • pp.23-29
    • /
    • 2021
  • Recent deep learning-based natural language processing studies are conducting research to improve performance by training large amounts of data from various sources together. However, there is a possibility that the methodology of learning by combining data from various sources into one may prevent performance improvement. In the case of machine translation, data deviation occurs due to differences in translation(liberal, literal), style(colloquial, written, formal, etc.), domains, etc. Combining these corpora into one for learning can adversely affect performance. In this paper, we propose a new Corpus Weight Balance(CWB) method that considers the balance between parallel corpora in machine translation. As a result of the experiment, the model trained with balanced corpus showed better performance than the existing model. In addition, we propose an additional corpus construction process that enables coexistence with the human translation market, which can build high-quality parallel corpus even with a monolingual corpus.

Some considerations for construction of spontaneous speech/text corpus (자유발화음성 및 텍스트코퍼스 구축에 관한 검토)

  • 이용주
    • Proceedings of the Acoustical Society of Korea Conference
    • /
    • 1994.06c
    • /
    • pp.303-309
    • /
    • 1994
  • 최근의 음성연구의 관신은 낭독음성에서 자유발화음성으로 옮겨가고 있다. 본고에서는 자유발화음성을 대상으로한 음성번역 및 대화시스템의 연구동향과 함께 자유발화의 음성 및 텍스트코퍼스 구축을 위한 몇몇 사항들을 살펴보고, 필자들이 현재 수집중인 코퍼스의 예를 소개한다.

  • PDF

Is Compared to Different from Compared with? A Discussion of Prepositions that Are Particularly Difficult for EFL Learners

  • Lee, Seung-Ah
    • Journal of English Language & Literature
    • /
    • v.55 no.6
    • /
    • pp.1057-1085
    • /
    • 2009
  • This paper addresses the question of why prepositions are particularly difficult forEFL learners. The first reason for such difficulty lies in the distinction between seemingly equivalent prepositions such as to and with, as in compared to and compared with. Most monolingual learners' dictionaries regard these two phrases as virtually synonymous. Yet, the results of the corpus analysis conducted in this study indicate that there are differences between the two. A second reason why EFL learners have problems with prepositions is that there are often variations in the inputdata. For example, although from generally follows different, in American English different than is also used. On the other hand, in British English, different to is the second most commonly used construction. This type of regional variation, confirmed in the corpus findings of the present paper, causes confusion in students of English. A learner who is not accustomed to British English may be puzzled by the expression different to. Finally, L1 negative transfer is responsible for the incorrect use of expressions such as discuss about. An error of this sort is the result of interference from the learner's mother tongue. The English verb discuss is not subcategorized for a preposition, whereas the equivalent Korean verb, for example, requires a noun phrase combined with the postposition.

A Study in Design and Construction of Structured Documents for Dialogue Corpus (대화형 코퍼스의 설계 및 구조적 문서화에 관한 연구)

  • Kang Chang-Qui;Nam Myung-Woo;Yang Ok-Yul
    • The Journal of the Korea Contents Association
    • /
    • v.4 no.4
    • /
    • pp.1-10
    • /
    • 2004
  • Dialogue speech corpora that contain sufficient dialogue speech features are needed for performance assessment of a spoken language dialogue system. And labeling information of dialogue speech corpora plays an important role for improvement of recognition rate in acoustic and language models. In this paper, we examine the methods by which labeling information of dialogue speech corpora can be structured. More specifically, we examined how to represent features of dialogue speech in a structured document based XML and how to design the repository system of the information.

  • PDF

An Analysis of the Intellectual Structure of Venture-Creation Studies to build an Entrepreneurship Ontology (창업 온톨로지 구축을 위한 벤처창업 연구의 지식구조 분석)

  • Sim, Jae-Hu;Choi, Myeonggil
    • Knowledge Management Research
    • /
    • v.14 no.4
    • /
    • pp.75-86
    • /
    • 2013
  • The deeping interests and research toward Entrepreneurship, which is considered as an potential alternative for solving the continuing economic recession in the $21^{st}$ century, have grown. The process and methodology of the research could not be systematically arranged and the results of the research lack in efforts on the application of increasing suceess ratio in starting new business. This study adopted corpus methodology, through which we try to analyzes the knowledge structure in entrepreneurship research, derive essential concepts and the consisting domains in venture research. Based on the results of analysis, this study constructs the knowledge structure of venture research in a form of knowledge ontology. The results of the study could be a ground for entrepreneurship research and utilized as implication for a creation of construction for the entrepreneurship knowledge ontology.

  • PDF

An Intonation Study of Predicate ending in Current Korean - From final endings of ${\ulcorner}$-a/e, $t{\int}ijo$${\lrcorner}$ and ${\ulcorner}$p/simnida${\lrcorner}$ - (현대 서울말 평서문에 나타나는 억양 연구 - 어말어미 "-아/어, -지요" 와 "-ㅂ/습니다" 를 중심으로 -)

  • Yu, Ki-Won
    • Proceedings of the KSPS conference
    • /
    • 2005.04a
    • /
    • pp.3-7
    • /
    • 2005
  • This research is for finding prototypes and characteristics of intonation found in ${\ulcorner}$-a/e, $t{\int}ijo$<${\lrcorner}$ and ${\ulcorner}$p/simnida${\lrcorner}$ among modern Korean predicate statements by constructing spoken corpus based on the current radio broadcast. So the result of the study is as follows. : (1) The construction of the balanced spoken corpus and the standard for boundary determination of rhythm are needed for the intonation model of speech synthesis. (2) Korean intonation units have the splited word tone which includes the nuclear tone and the pre-nuclear tone makes unclear tone more detailed. (3) I made man and woman intonation models individually through t-test of SPSS. (4) The standard intonation model is devided '-ajo'type and '-nida'type

  • PDF

Ordering a Left-branching Language: Heaviness vs. Givenness

  • Choi, Hye-Won
    • Language and Information
    • /
    • v.13 no.1
    • /
    • pp.39-56
    • /
    • 2009
  • This paper investigates ordering alternation phenomena in Korean using the dative construction data from Sejong Corpus of Modern Korean (Kim, 2000). The paper first shows that syntactic weight and information structure are distinct and independent factors that influence word order in Korean. Moreover, it reveals that heaviness and givenness compete each other and exert diverging effects on word order, which contrasts the converging effects of these factors shown in word orders of right-branching languages like English. The typological variation of syntactic weight effect poses interesting theoretical and empirical questions, which are discussed in relation to processing efficiency in ordering.

  • PDF

Copy Raising Construction in English: A Usage-based Perspective

  • Kim, Jong-Bok
    • Language and Information
    • /
    • v.16 no.2
    • /
    • pp.1-15
    • /
    • 2012
  • In accounting for the so-called copy raising (CR) in English, the movement perspective has assumed that the embedded subject of the CR verb's sentential complement is raised to the matrix subject, leaving behind its pronominal copy. This kind of movement-based analysis raises both empirical and analytical issues, when considering variations in the pronominal copy constraint. This paper investigates the actual uses of the construction, using online-available corpora. Based on this corpus search, we classify two different types of copy raising predicates (genuine and perception), and discuss their grammatical properties in detail. We suggest that the simple copying rule couched upon movement operations is not enough to capture great variations in the uses of the construction, and show that interpretive constraints, e.g., perceptual characterization condition, play an important role in licensing the construction.

  • PDF

Automatic Construction of Korean Two-level Lexicon using Lexical and Morphological Information (어휘 및 형태 정보를 이용한 한국어 Two-level 어휘사전 자동 구축)

  • Kim, Bogyum;Lee, Jae Sung
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.12
    • /
    • pp.865-872
    • /
    • 2013
  • Two-level morphology analysis method is one of rule-based morphological analysis method. This approach handles morphological transformation using rules and analyzes words with morpheme connection information in a lexicon. It is independent of language and Korean Two-level system was also developed. But, it was limited in practical use, because of using very small set of lexicon built manually. And it has also a over-generation problem. In this paper, we propose an automatic construction method of Korean Two-level lexicon for PC-KIMMO from morpheme tagged corpus. We also propose a method to solve over-generation problem using lexical information and sub-tags. The experiment showed that the proposed method reduced over-generation by 68% compared with the previous method, and the performance increased from 39% to 65% in f-measure.