• Title/Summary/Keyword: 자연어 분석

Search Result 559, Processing Time 0.023 seconds

Building a Korean Sentiment Lexicon Using Collective Intelligence (집단지성을 이용한 한글 감성어 사전 구축)

  • An, Jungkook;Kim, Hee-Woong
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.2
    • /
    • pp.49-67
    • /
    • 2015
  • Recently, emerging the notion of big data and social media has led us to enter data's big bang. Social networking services are widely used by people around the world, and they have become a part of major communication tools for all ages. Over the last decade, as online social networking sites become increasingly popular, companies tend to focus on advanced social media analysis for their marketing strategies. In addition to social media analysis, companies are mainly concerned about propagating of negative opinions on social networking sites such as Facebook and Twitter, as well as e-commerce sites. The effect of online word of mouth (WOM) such as product rating, product review, and product recommendations is very influential, and negative opinions have significant impact on product sales. This trend has increased researchers' attention to a natural language processing, such as a sentiment analysis. A sentiment analysis, also refers to as an opinion mining, is a process of identifying the polarity of subjective information and has been applied to various research and practical fields. However, there are obstacles lies when Korean language (Hangul) is used in a natural language processing because it is an agglutinative language with rich morphology pose problems. Therefore, there is a lack of Korean natural language processing resources such as a sentiment lexicon, and this has resulted in significant limitations for researchers and practitioners who are considering sentiment analysis. Our study builds a Korean sentiment lexicon with collective intelligence, and provides API (Application Programming Interface) service to open and share a sentiment lexicon data with the public (www.openhangul.com). For the pre-processing, we have created a Korean lexicon database with over 517,178 words and classified them into sentiment and non-sentiment words. In order to classify them, we first identified stop words which often quite likely to play a negative role in sentiment analysis and excluded them from our sentiment scoring. In general, sentiment words are nouns, adjectives, verbs, adverbs as they have sentimental expressions such as positive, neutral, and negative. On the other hands, non-sentiment words are interjection, determiner, numeral, postposition, etc. as they generally have no sentimental expressions. To build a reliable sentiment lexicon, we have adopted a concept of collective intelligence as a model for crowdsourcing. In addition, a concept of folksonomy has been implemented in the process of taxonomy to help collective intelligence. In order to make up for an inherent weakness of folksonomy, we have adopted a majority rule by building a voting system. Participants, as voters were offered three voting options to choose from positivity, negativity, and neutrality, and the voting have been conducted on one of the largest social networking sites for college students in Korea. More than 35,000 votes have been made by college students in Korea, and we keep this voting system open by maintaining the project as a perpetual study. Besides, any change in the sentiment score of words can be an important observation because it enables us to keep track of temporal changes in Korean language as a natural language. Lastly, our study offers a RESTful, JSON based API service through a web platform to make easier support for users such as researchers, companies, and developers. Finally, our study makes important contributions to both research and practice. In terms of research, our Korean sentiment lexicon plays an important role as a resource for Korean natural language processing. In terms of practice, practitioners such as managers and marketers can implement sentiment analysis effectively by using Korean sentiment lexicon we built. Moreover, our study sheds new light on the value of folksonomy by combining collective intelligence, and we also expect to give a new direction and a new start to the development of Korean natural language processing.

Investigating Major Topics Through the Analysis of Depression-related Facebook Group Posts (페이스북 그룹 게시물 분석을 통한 우울증 관련 주제에 대한 고찰)

  • Zhu, Yongjun;Kim, Donghun;Lee, Changho;Lee, Yongjeong
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.53 no.4
    • /
    • pp.171-187
    • /
    • 2019
  • The study aims to analyze the posts of depression-related Facebook groups to understand major topics discussed by group users. Specifically, the purpose of the study is to identify the topics and keywords of the posts to understand what users discuss about depression. Depression is a mental disorder that is somewhat sensitive in the online community, which is characterized by accessibility, openness and anonymity. The researchers have implemented a natural language-based data analysis framework that includes components ranging from Facebook data collection to the automated extraction of topics. Using the framework, we collected and analyzed 885 posts created in the past one year from the largest Facebook depression group. To derive more complete and accurate topics, we combined both automated and manual (e.g., stop words removal, topic size determination) methods. Results indicate that users discuss a variety of topics including depression in general, human relations, mood and feeling, depression symptoms, suicide, medical references, family and etc.

Attention-based word correlation analysis system for big data analysis (빅데이터 분석을 위한 어텐션 기반의 단어 연관관계 분석 시스템)

  • Chi-Gon, Hwang;Chang-Pyo, Yoon;Soo-Wook, Lee
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.27 no.1
    • /
    • pp.41-46
    • /
    • 2023
  • Recently, big data analysis can use various techniques according to the development of machine learning. Big data collected in reality lacks an automated refining technique for the same or similar terms based on semantic analysis of the relationship between words. Since most of the big data is described in general sentences, it is difficult to understand the meaning and terms of the sentences. To solve these problems, it is necessary to understand the morphological analysis and meaning of sentences. Accordingly, NLP, a technique for analyzing natural language, can understand the word's relationship and sentences. Among the NLP techniques, the transformer has been proposed as a way to solve the disadvantages of RNN by using self-attention composed of an encoder-decoder structure of seq2seq. In this paper, transformers are used as a way to form associations between words in order to understand the words and phrases of sentences extracted from big data.

Extension of Z Schema for Component Formal Specification (컴포넌트 정형명세를 위한 Z 스키마의 확장)

  • 이재희;장종표;김병기
    • Proceedings of the Korea Multimedia Society Conference
    • /
    • 2002.05d
    • /
    • pp.661-664
    • /
    • 2002
  • 컴포넌트를 개발하는데 있어서 컴포넌트 명세의 정확성과 명세의 검증을 통하여 에러를 찾아 낸다는 것은 컴포넌트의 전체 품질에 매우 중요한 의미를 갖는다. 그러나, 기존의 컴포넌트 명세는 구문적인 측면은 잘 정의하고 있지만, 의미적인 측면은 자연어를 사용하여 모순과 모호성이 흔히 발생한다. 컴포넌트 명세에 있어서 정형적 문법을 사용할 경우 이러한 모호성을 제거함으로써 명세 오류들을 매우 효과적으로 줄여준다. 본 논문에서는 컴포넌트의 품질을 높일 수 있도록 분석력과 논리성이 검증된 정형 명세 언어 Z의 스키마 확장을 이용하여 컴포넌트를 명세하므로써 컴포넌트 구현 및 사용상의 오류를 분석할 수 있는 방법을 제안한다.

  • PDF

A Korean Language Stemmer based on Unsupervised Learning (자율 학습에 의한 실질 형태소와 형식 형태소의 분리)

  • Cha, Yong-Tae;Cho, Se-Hyeong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.11a
    • /
    • pp.577-580
    • /
    • 2002
  • 자연어의 처리를 위해 반드시 필요한 형태소 분석에는 여러 가지 방법이 있으나 기본적으로 사전을 갖춘 상태에서 가장 가능성 있는 후보를 선택하는 방식을 선택한다. 이러한 방식으로는 사전이 없는 미지의 언어를 분석하기는 불가능하다. 기지의 언어라도 지속적으로 어휘가 변하는 경우나 매우 특별한 분야의 경우에는 필요로 하는 사전이 존재하지 않는다. 본 논문에서는 태그가 없는 단순 말뭉치만을 가지고 자율학습을 이용하여 한국어의 실질 형태소와 형식 형태소를 분리해내는 기법에 대하여 기술한다.

  • PDF

Gender classification of Korean drama script lines using KoBERT (KoBERT를 활용한 한국 드라마 대본 대사 성별 구분)

  • Se-Hui Yi;Gum-Kyu Sun
    • Annual Conference on Human and Language Technology
    • /
    • 2022.10a
    • /
    • pp.470-472
    • /
    • 2022
  • 최근 글로벌 OTT 서비스에서 한국드라마가 세계적 인기를 얻음에 따라 드라마 콘텐츠의 가치가 높아지고 있다. 드라마 대본은 드라마 제작에 있어서 핵심이 되는 데이터로, 특히 대사에는 인물의 특성이 잘 나타나 있다. 본 논문에서는 KoBERT 모델을 활용해 드라마 대사에서 인물의 특성 중 하나인 성별을 구분하고 실험 결과를 제시한다. KoBERT 모델로 대사의 성별을 분류한 뒤, 콘텐츠 분석과 인공지능 창작 측면에서의 활용 가능성에 대해 논의한다.

  • PDF

Natural Language Toolkit _ Korean (NLTKo 1.0: 한국어 언어처리 도구)

  • Hong, Seong-Tae;Cha, Jeong-Won
    • Annual Conference on Human and Language Technology
    • /
    • 2021.10a
    • /
    • pp.554-557
    • /
    • 2021
  • NLTKo는 한국어 분석 도구들을 NLTK에 결합하여 사용할 수 있게 만든 도구이다. NLTKo는 전처리 도구, 토크나이저, 형태소 분석기, 세종 의미사전, 분류 및 기계번역 성능 평가 도구를 추가로 제공한다. 이들은 기존의 NLTK 함수와 동일한 방법으로 사용할 수 있도록 구현하였다. 또한 세종 의미사전을 제공하여 한국어 동의어/반의어, 상/하위어 등을 제공한다. NLTKo는 한국어 자연어처리를 위한 교육에 도움이 될 것으로 믿는다.

  • PDF

An Example-Based Natural Language Dialogue System for EPG Information Access (EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템)

  • Kim, Seok-Hwan;Lee, Cheong-Jae;Jung, Sang-Keun;Lee, GaryGeun-Bae
    • Journal of KIISE:Software and Applications
    • /
    • v.34 no.2
    • /
    • pp.123-130
    • /
    • 2007
  • In this paper, we present an example-based natural language dialogue system for Electronic Program Guide Information Access. We introduce an effective and practical dialogue management technique incorporating dialogue examples and situation-based rules. In order to generate cooperative responses to smoothly lead the dialogue with users, our natural language dialogue system consists of natural language understanding, dialogue manager, system utterance generator. and EPG database manager. Each module is designed and implemented to make an effective and practical natural language dialogue system. In particular, in order to reflect the up-to-date EPG information which is updated frequently and periodically, we applied a web-mining technology to the EPG database manager, which builds the content database based on automatically extracted information from popular EPG websites. The automatically generated content database is used by other modules in the system for building their own resources. Evaluations show that our system performs EPG access task in high performance and can be managed with low cost.

Statistical Approach to Sentiment Classification using MapReduce (맵리듀스를 이용한 통계적 접근의 감성 분류)

  • Kang, Mun-Su;Baek, Seung-Hee;Choi, Young-Sik
    • Science of Emotion and Sensibility
    • /
    • v.15 no.4
    • /
    • pp.425-440
    • /
    • 2012
  • As the scale of the internet grows, the amount of subjective data increases. Thus, A need to classify automatically subjective data arises. Sentiment classification is a classification of subjective data by various types of sentiments. The sentiment classification researches have been studied focused on NLP(Natural Language Processing) and sentiment word dictionary. The former sentiment classification researches have two critical problems. First, the performance of morpheme analysis in NLP have fallen short of expectations. Second, it is not easy to choose sentiment words and determine how much a word has a sentiment. To solve these problems, this paper suggests a combination of using web-scale data and a statistical approach to sentiment classification. The proposed method of this paper is using statistics of words from web-scale data, rather than finding a meaning of a word. This approach differs from the former researches depended on NLP algorithms, it focuses on data. Hadoop and MapReduce will be used to handle web-scale data.

  • PDF

Intelligent Information Retrieval Using Interactive Query Processing Agent (대화형 질의 처리 에이전트를 이용한 지능형 정보검색)

  • 이현영;이기오;한용기
    • Journal of the Korea Computer Industry Society
    • /
    • v.4 no.12
    • /
    • pp.901-910
    • /
    • 2003
  • Generally, most commercial retrieval engines adopt boolean query as user's query type. Although boolean query is useful to retrieval engines that need fast retrieval, it is not easy for user to express his demands with boolean operators. So, many researches have been studied for decades about information retrieval systems using natural language query that is convenient for user. To retrieve documents that are suitable for user's demands, they have to express their demands correctly, So, this thesis proposes interactive query process agent using natural language. This agent expresses demands concrete through gradual interaction with user, When users input a natural language Query, this agent analyzes the query and generates boolean query by selecting proper keyword and feedbacks the state of the keyword selected. If the keyword is a synonymy or a polysemy, the agent expands or limits the keyword through interaction with user. It makes user express demands more concrete and improve system performance. So, this agent can improve the precision of Information Retrieval.

  • PDF