• Title/Summary/Keyword: Lexicon

Search Result 273, Processing Time 0.029 seconds

Identification of Profane Words in Cyberbullying Incidents within Social Networks

  • Ali, Wan Noor Hamiza Wan;Mohd, Masnizah;Fauzi, Fariza
    • Journal of Information Science Theory and Practice
    • /
    • v.9 no.1
    • /
    • pp.24-34
    • /
    • 2021
  • The popularity of social networking sites (SNS) has facilitated communication between users. The usage of SNS helps users in their daily life in various ways such as sharing of opinions, keeping in touch with old friends, making new friends, and getting information. However, some users misuse SNS to belittle or hurt others using profanities, which is typical in cyberbullying incidents. Thus, in this study, we aim to identify profane words from the ASKfm corpus to analyze the profane word distribution across four different roles involved in cyberbullying based on lexicon dictionary. These four roles are: harasser, victim, bystander that assists the bully, and bystander that defends the victim. Evaluation in this study focused on occurrences of the profane word for each role from the corpus. The top 10 common words used in the corpus are also identified and represented in a graph. Results from the analysis show that these four roles used profane words in their conversation with different weightage and distribution, even though the profane words used are mostly similar. The harasser is the first ranked that used profane words in the conversation compared to other roles. The results can be further explored and considered as a potential feature in a cyberbullying detection model using a machine learning approach. Results in this work will contribute to formulate the suitable representation. It is also useful in modeling a cyberbullying detection model based on the identification of profane word distribution across different cyberbullying roles in social networks for future works.

Performance of Korean spontaneous speech recognizers based on an extended phone set derived from acoustic data (음향 데이터로부터 얻은 확장된 음소 단위를 이용한 한국어 자유발화 음성인식기의 성능)

  • Bang, Jeong-Uk;Kim, Sang-Hun;Kwon, Oh-Wook
    • Phonetics and Speech Sciences
    • /
    • v.11 no.3
    • /
    • pp.39-47
    • /
    • 2019
  • We propose a method to improve the performance of spontaneous speech recognizers by extending their phone set using speech data. In the proposed method, we first extract variable-length phoneme-level segments from broadcast speech signals, and convert them to fixed-length latent vectors using an long short-term memory (LSTM) classifier. We then cluster acoustically similar latent vectors and build a new phone set by choosing the number of clusters with the lowest Davies-Bouldin index. We also update the lexicon of the speech recognizer by choosing the pronunciation sequence of each word with the highest conditional probability. In order to analyze the acoustic characteristics of the new phone set, we visualize its spectral patterns and segment duration. Through speech recognition experiments using a larger training data set than our own previous work, we confirm that the new phone set yields better performance than the conventional phoneme-based and grapheme-based units in both spontaneous speech recognition and read speech recognition.

Developing a Korean sentiment lexicon through BPE (BPE를 활용한 한국어 감정사전 제작)

  • Park, Ho-Min;Cheon, Min-Ah;Nam-Goong, Young;Choi, Min-Seok;Yoon, Ho;Kim, Jae-Kyun;Kim, Jae-Hoon
    • Annual Conference on Human and Language Technology
    • /
    • 2019.10a
    • /
    • pp.510-513
    • /
    • 2019
  • 감정분석은 텍스트에서 나타난 저자 혹은 발화자의 태도, 의견 등과 같은 주관적인 정보를 추출하는 기술이며, 여론 분석, 시장 동향 분석 등 다양한 분야에 두루 사용된다. 감정분석 방법은 사전 기반 방법, 기계학습 기반 방법 등이 있다. 본 논문은 사전 기반 감정분석에 필요한 한국어 감정사전 자동 구축 방법을 제안한다. 본 논문은 영어 감정사전으로부터 한국어 감정사전을 자동으로 구축하는 방법이며, 크게 세 단계로 구성된다. 첫 번째는 한영 병렬 말뭉치를 이용한 한영 이중언어 사전을 구축하는 단계이고, 두 번째는 한영 이중언어 사전을 통한 한영 이중언어 그래프를 생성하는 단계이며, 세 번째는 영어 단어의 감정값을 한국어 BPE의 감정값으로 전파하는 단계이다. 본 논문에서는 제안된 방법의 유효성을 보이기 위해 사전 기반 한국어 감정분석 시스템을 구축하여 평가하였으며, 그 결과 제안된 방법이 합리적인 방법임을 확인할 수 있었으며 향후 연구를 통해 개선한다면 질 좋은 한국어 감정사전을 효과적인 방법으로 구축할 수 있을 것이다.

  • PDF

Developing the Automated Sentiment Learning Algorithm to Build the Korean Sentiment Lexicon for Finance (재무분야 감성사전 구축을 위한 자동화된 감성학습 알고리즘 개발)

  • Su-Ji Cho;Ki-Kwang Lee;Cheol-Won Yang
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.46 no.1
    • /
    • pp.32-41
    • /
    • 2023
  • Recently, many studies are being conducted to extract emotion from text and verify its information power in the field of finance, along with the recent development of big data analysis technology. A number of prior studies use pre-defined sentiment dictionaries or machine learning methods to extract sentiment from the financial documents. However, both methods have the disadvantage of being labor-intensive and subjective because it requires a manual sentiment learning process. In this study, we developed a financial sentiment dictionary that automatically extracts sentiment from the body text of analyst reports by using modified Bayes rule and verified the performance of the model through a binary classification model which predicts actual stock price movements. As a result of the prediction, it was found that the proposed financial dictionary from this research has about 4% better predictive power for actual stock price movements than the representative Loughran and McDonald's (2011) financial dictionary. The sentiment extraction method proposed in this study enables efficient and objective judgment because it automatically learns the sentiment of words using both the change in target price and the cumulative abnormal returns. In addition, the dictionary can be easily updated by re-calculating conditional probabilities. The results of this study are expected to be readily expandable and applicable not only to analyst reports, but also to financial field texts such as performance reports, IR reports, press articles, and social media.

Semantic analysis of unstructured information considering the step in progress of water quality accidents in the water supply systems (상수도시스템 수질사고의 전개양상을 고려한 비정형정보 의미분석)

  • Hong, Sungjin;Moon, Gihoon;Yang, Seong Hun;Yoo, Do Guen
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.378-378
    • /
    • 2022
  • 상수도시스템의 과정 중 최종 단계인 급수단계에서 지역전반에 수질문제가 발생할 경우, 직간접적인 피해의 해결은 장기간 지속될 수 있다. 본 연구에서는 실시간 비정형정보의 빅데이터 분석을 통해 상수도시스템에서 수질사고 문제의 파급력과 2차 피해 등의 연결 관계 변화 추적을 위한 기초적 분석을 수행하였다. 과거 대규모 수질사고가 발생된 바 있는 인천광역시 유충발생 사고를 대상으로 뉴스 기사 웹크롤링 절차를 정립하고, 그 결과를 분석하였다. '인천 유충'이 최초 보도되었던 2020년 7월 13일 부터 이후 1년을 대상으로 네이버 통합검색에 의해 표출되는 뉴스기사를 웹크롤링하였으며, 프로그래밍을 통한 불용어 제거 및 관련성 검토를 통해 총 920건의 기사를 분석하였다. 수질사고의 전개양상에 따라 사고발생, 확산, 수습, 그리고 보상의 4단계로 임의 구분하여 분석하였다. 의미분석을 위한 토픽모델링 기법은 잠재 디리클레 할당(Latent Dirichlet Allocation, LDA) 방법을 적용하였으며, 긍부정 감정분석은 KNU 한국어 감성사전(KNU sentiment lexicon)을 활용하여 수행하였다. 토픽 모델링 결과, 사고 발생에서부터 확산, 수습, 보상의 단계에 맞춰 적절한 주제어의 조합에 따른 기사들이 도출되었으며, 단계별 긍부정 기사 비율역시 사고의 전개단계에 따라 적절히 나타남을 확인하였다. 제시된 수질사고 관련 비정형정보 분석 방법론과 결과는 과거 사고 사례 분석을 통한 검색 및 긍부정 키워드 확정, 키워드 발생 비율 변동(사고전과 후)에 따른 상황판단 기준설정 등에 활용이 가능하다.

  • PDF

Anatomy of Sentiment Analysis of Tweets Using Machine Learning Approach

  • Misbah Iram;Saif Ur Rehman;Shafaq Shahid;Sayeda Ambreen Mehmood
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.10
    • /
    • pp.97-106
    • /
    • 2023
  • Sentiment analysis using social network platforms such as Twitter has achieved tremendous results. Twitter is an online social networking site that contains a rich amount of data. The platform is known as an information channel corresponding to different sites and categories. Tweets are most often publicly accessible with very few limitations and security options available. Twitter also has powerful tools to enhance the utility of Twitter and a powerful search system to make publicly accessible the recently posted tweets by keyword. As popular social media, Twitter has the potential for interconnectivity of information, reviews, updates, and all of which is important to engage the targeted population. In this work, numerous methods that perform a classification of tweet sentiment in Twitter is discussed. There has been a lot of work in the field of sentiment analysis of Twitter data. This study provides a comprehensive analysis of the most standard and widely applicable techniques for opinion mining that are based on machine learning and lexicon-based along with their metrics. The proposed work is helpful to analyze the information in the tweets where opinions are highly unstructured, heterogeneous, and polarized positive, negative or neutral. In order to validate the performance of the proposed framework, an extensive series of experiments has been performed on the real world twitter dataset that alter to show the effectiveness of the proposed framework. This research effort also highlighted the recent challenges in the field of sentiment analysis along with the future scope of the proposed work.

Performance Comparison and Error Analysis of Korean Bio-medical Named Entity Recognition (한국어 생의학 개체명 인식 성능 비교와 오류 분석)

  • Jae-Hong Lee
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.19 no.4
    • /
    • pp.701-708
    • /
    • 2024
  • The advent of transformer architectures in deep learning has been a major breakthrough in natural language processing research. Object name recognition is a branch of natural language processing and is an important research area for tasks such as information retrieval. It is also important in the biomedical field, but the lack of Korean biomedical corpora for training has limited the development of Korean clinical research using AI. In this study, we built a new biomedical corpus for Korean biomedical entity name recognition and selected language models pre-trained on a large Korean corpus for transfer learning. We compared the name recognition performance of the selected language models by F1-score and the recognition rate by tag, and analyzed the errors. In terms of recognition performance, KlueRoBERTa showed relatively good performance. The error analysis of the tagging process shows that the recognition performance of Disease is excellent, but Body and Treatment are relatively low. This is due to over-segmentation and under-segmentation that fails to properly categorize entity names based on context, and it will be necessary to build a more precise morphological analyzer and a rich lexicon to compensate for the incorrect tagging.

Usefulness of Three-Dimensional Maximal Intensity Projection (MIP) Reconstruction Image in Breast MRI (유방자기공명영상에서 3 차원 최대 강도 투사 재건 영상의 유용성)

  • Kim, Hyun-Sung;Kang, Bong-Joo;Kim, Sung-Hun;Choi, Jae-Jeong;Lee, Ji-Hye
    • Investigative Magnetic Resonance Imaging
    • /
    • v.13 no.2
    • /
    • pp.183-189
    • /
    • 2009
  • Purpose : To evaluate the usefulness of three-dimensional (3D) maximal intensity projection (MIP) reconstruction method in breast MRI. Materials and Methods : Total 54 breasts of consecutive 27 patients were examined by breast MRI. Breast MRI was performed using GE Signa Excite Twin speed (GE medical system, Wisconsin, USA) 1.5T. We obtained routine breast MR images including axial T2WI, T1WI, sagittal T1FS, dynamic contrast-enhanced T1FS, and subtraction images. 3D MIP reconstruction images were obtained as follows; subtraction images were obtained using TIPS and early stage of contrast-enhanced TIPS images. And then 3D MIP images were obtained using the subtraction images through advantage workstation (GE Medical system). We detected and analyzed the lesions in the 3D MIP and routine MRI images according to ACR $BIRADS^{(R)}$ MRI lexicon. And then we compared the findings of 3D MIP and those of routine breast MR images and evaluated whether 3D MIP had additional information comparing to routine MR images. Results : 3D MIP images detect the 43 of 56 masses found on routine MR images (76.8%). In non-mass like enhancement, 3D MIP detected 17 of 20 lesions (85 %). And there were one hundred sixty nine foci at 3D MIP images and one hundred nine foci at routine MR images. 3D MIP images detected 14 of 23 category 3 lesions (60.9%), 11 of 16 category 4 lesions (68.87%), 28 of 28 Category 5 lesions (100%). In analyzing the enhancing lesions at 3D MIP images, assessment categories of the lesions were correlated as the results at routine MR images (p-value < 0.0001). 3D MIP detected additional two daughter nodules that were descriped foci at routine MR images and additional one nodule that was not detected at routine MR images. Conclusion : 3D MIP image has some limitations but is useful as additional image of routine breast MR Images.

  • PDF

Bankruptcy Prediction Modeling Using Qualitative Information Based on Big Data Analytics (빅데이터 기반의 정성 정보를 활용한 부도 예측 모형 구축)

  • Jo, Nam-ok;Shin, Kyung-shik
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.2
    • /
    • pp.33-56
    • /
    • 2016
  • Many researchers have focused on developing bankruptcy prediction models using modeling techniques, such as statistical methods including multiple discriminant analysis (MDA) and logit analysis or artificial intelligence techniques containing artificial neural networks (ANN), decision trees, and support vector machines (SVM), to secure enhanced performance. Most of the bankruptcy prediction models in academic studies have used financial ratios as main input variables. The bankruptcy of firms is associated with firm's financial states and the external economic situation. However, the inclusion of qualitative information, such as the economic atmosphere, has not been actively discussed despite the fact that exploiting only financial ratios has some drawbacks. Accounting information, such as financial ratios, is based on past data, and it is usually determined one year before bankruptcy. Thus, a time lag exists between the point of closing financial statements and the point of credit evaluation. In addition, financial ratios do not contain environmental factors, such as external economic situations. Therefore, using only financial ratios may be insufficient in constructing a bankruptcy prediction model, because they essentially reflect past corporate internal accounting information while neglecting recent information. Thus, qualitative information must be added to the conventional bankruptcy prediction model to supplement accounting information. Due to the lack of an analytic mechanism for obtaining and processing qualitative information from various information sources, previous studies have only used qualitative information. However, recently, big data analytics, such as text mining techniques, have been drawing much attention in academia and industry, with an increasing amount of unstructured text data available on the web. A few previous studies have sought to adopt big data analytics in business prediction modeling. Nevertheless, the use of qualitative information on the web for business prediction modeling is still deemed to be in the primary stage, restricted to limited applications, such as stock prediction and movie revenue prediction applications. Thus, it is necessary to apply big data analytics techniques, such as text mining, to various business prediction problems, including credit risk evaluation. Analytic methods are required for processing qualitative information represented in unstructured text form due to the complexity of managing and processing unstructured text data. This study proposes a bankruptcy prediction model for Korean small- and medium-sized construction firms using both quantitative information, such as financial ratios, and qualitative information acquired from economic news articles. The performance of the proposed method depends on how well information types are transformed from qualitative into quantitative information that is suitable for incorporating into the bankruptcy prediction model. We employ big data analytics techniques, especially text mining, as a mechanism for processing qualitative information. The sentiment index is provided at the industry level by extracting from a large amount of text data to quantify the external economic atmosphere represented in the media. The proposed method involves keyword-based sentiment analysis using a domain-specific sentiment lexicon to extract sentiment from economic news articles. The generated sentiment lexicon is designed to represent sentiment for the construction business by considering the relationship between the occurring term and the actual situation with respect to the economic condition of the industry rather than the inherent semantics of the term. The experimental results proved that incorporating qualitative information based on big data analytics into the traditional bankruptcy prediction model based on accounting information is effective for enhancing the predictive performance. The sentiment variable extracted from economic news articles had an impact on corporate bankruptcy. In particular, a negative sentiment variable improved the accuracy of corporate bankruptcy prediction because the corporate bankruptcy of construction firms is sensitive to poor economic conditions. The bankruptcy prediction model using qualitative information based on big data analytics contributes to the field, in that it reflects not only relatively recent information but also environmental factors, such as external economic conditions.

Methodology for Identifying Issues of User Reviews from the Perspective of Evaluation Criteria: Focus on a Hotel Information Site (사용자 리뷰의 평가기준 별 이슈 식별 방법론: 호텔 리뷰 사이트를 중심으로)

  • Byun, Sungho;Lee, Donghoon;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.3
    • /
    • pp.23-43
    • /
    • 2016
  • As a result of the growth of Internet data and the rapid development of Internet technology, "big data" analysis has gained prominence as a major approach for evaluating and mining enormous data for various purposes. Especially, in recent years, people tend to share their experiences related to their leisure activities while also reviewing others' inputs concerning their activities. Therefore, by referring to others' leisure activity-related experiences, they are able to gather information that might guarantee them better leisure activities in the future. This phenomenon has appeared throughout many aspects of leisure activities such as movies, traveling, accommodation, and dining. Apart from blogs and social networking sites, many other websites provide a wealth of information related to leisure activities. Most of these websites provide information of each product in various formats depending on different purposes and perspectives. Generally, most of the websites provide the average ratings and detailed reviews of users who actually used products/services, and these ratings and reviews can actually support the decision of potential customers in purchasing the same products/services. However, the existing websites offering information on leisure activities only provide the rating and review based on one stage of a set of evaluation criteria. Therefore, to identify the main issue for each evaluation criterion as well as the characteristics of specific elements comprising each criterion, users have to read a large number of reviews. In particular, as most of the users search for the characteristics of the detailed elements for one or more specific evaluation criteria based on their priorities, they must spend a great deal of time and effort to obtain the desired information by reading more reviews and understanding the contents of such reviews. Although some websites break down the evaluation criteria and direct the user to input their reviews according to different levels of criteria, there exist excessive amounts of input sections that make the whole process inconvenient for the users. Further, problems may arise if a user does not follow the instructions for the input sections or fill in the wrong input sections. Finally, treating the evaluation criteria breakdown as a realistic alternative is difficult, because identifying all the detailed criteria for each evaluation criterion is a challenging task. For example, if a review about a certain hotel has been written, people tend to only write one-stage reviews for various components such as accessibility, rooms, services, or food. These might be the reviews for most frequently asked questions, such as distance between the nearest subway station or condition of the bathroom, but they still lack detailed information for these questions. In addition, in case a breakdown of the evaluation criteria was provided along with various input sections, the user might only fill in the evaluation criterion for accessibility or fill in the wrong information such as information regarding rooms in the evaluation criteria for accessibility. Thus, the reliability of the segmented review will be greatly reduced. In this study, we propose an approach to overcome the limitations of the existing leisure activity information websites, namely, (1) the reliability of reviews for each evaluation criteria and (2) the difficulty of identifying the detailed contents that make up the evaluation criteria. In our proposed methodology, we first identify the review content and construct the lexicon for each evaluation criterion by using the terms that are frequently used for each criterion. Next, the sentences in the review documents containing the terms in the constructed lexicon are decomposed into review units, which are then reconstructed by using the evaluation criteria. Finally, the issues of the constructed review units by evaluation criteria are derived and the summary results are provided. Apart from the derived issues, the review units are also provided. Therefore, this approach aims to help users save on time and effort, because they will only be reading the relevant information they need for each evaluation criterion rather than go through the entire text of review. Our proposed methodology is based on the topic modeling, which is being actively used in text analysis. The review is decomposed into sentence units rather than considering the whole review as a document unit. After being decomposed into individual review units, the review units are reorganized according to each evaluation criterion and then used in the subsequent analysis. This work largely differs from the existing topic modeling-based studies. In this paper, we collected 423 reviews from hotel information websites and decomposed these reviews into 4,860 review units. We then reorganized the review units according to six different evaluation criteria. By applying these review units in our methodology, the analysis results can be introduced, and the utility of proposed methodology can be demonstrated.