• Title/Summary/Keyword: TF-IDF analysis

Search Result 197, Processing Time 0.023 seconds

Analyzing data-related policy programs in Korea using text mining and network cluster analysis (텍스트 마이닝과 네트워크 군집 분석을 활용한 한국의 데이터 관련 정책사업 분석)

  • Sungjun Choi;Kiyoon Shin;Yoonhwan Oh
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.28 no.6
    • /
    • pp.63-81
    • /
    • 2023
  • This study endeavors to classify and categorize similar policy programs through network clustering analysis, using textual information from data-related policy programs in Korea. To achieve this, descriptions of data-related budgetary programs in South Korea in 2022 were collected, and keywords from the program contents were extracted. Subsequently, the similarity between each program was derived using TF-IDF, and policy program network was constructed accordingly. Following this, the structural characteristics of the network were analyzed, and similar policy programs were clustered and categorized through network clustering. Upon analyzing a total of 97 programs, 7 major clusters were identified, signifying that programs with analogous themes or objectives were categorized based on application area or services utilizing data. The findings of this research illuminate the current status of data-related policy programs in Korea, providing policy implications for a strategic approach to planning future national data strategies and programs, and contributing to the establishment of evidence-based policies.

Digital Transformation: Using D.N.A.(Data, Network, AI) Keywords Generalized DMR Analysis (디지털 전환: D.N.A.(Data, Network, AI) 키워드를 활용한 토픽 모델링)

  • An, Sehwan;Ko, Kangwook;Kim, Youngmin
    • Knowledge Management Research
    • /
    • v.23 no.3
    • /
    • pp.129-152
    • /
    • 2022
  • As a key infrastructure for digital transformation, the spread of data, network, artificial intelligence (D.N.A.) fields and the emergence of promising industries are laying the groundwork for active digital innovation throughout the economy. In this study, by applying the text mining methodology, major topics were derived by using the abstract, publication year, and research field of the study corresponding to the SCIE, SSCI, and A&HCI indexes of the WoS database as input variables. First, main keywords were identified through TF and TF-IDF analysis based on word appearance frequency, and then topic modeling was performed using g-DMR. With the advantage of the topic model that can utilize various types of variables as meta information, it was possible to properly explore the meaning beyond simply deriving a topic. According to the analysis results, topics such as business intelligence, manufacturing production systems, service value creation, telemedicine, and digital education were identified as major research topics in digital transformation. To summarize the results of topic modeling, 1) research on business intelligence has been actively conducted in all areas after COVID-19, and 2) issues such as intelligent manufacturing solutions and metaverses have emerged in the manufacturing field. It has been confirmed that the topic of production systems is receiving attention once again. Finally, 3) Although the topic itself can be viewed separately in terms of technology and service, it was found that it is undesirable to interpret it separately because a number of studies comprehensively deal with various services applied by combining the relevant technologies.

Research on Designing Korean Emotional Dictionary using Intelligent Natural Language Crawling System in SNS (SNS대상의 지능형 자연어 수집, 처리 시스템 구현을 통한 한국형 감성사전 구축에 관한 연구)

  • Lee, Jong-Hwa
    • The Journal of Information Systems
    • /
    • v.29 no.3
    • /
    • pp.237-251
    • /
    • 2020
  • Purpose The research was studied the hierarchical Hangul emotion index by organizing all the emotions which SNS users are thinking. As a preliminary study by the researcher, the English-based Plutchick (1980)'s emotional standard was reinterpreted in Korean, and a hashtag with implicit meaning on SNS was studied. To build a multidimensional emotion dictionary and classify three-dimensional emotions, an emotion seed was selected for the composition of seven emotion sets, and an emotion word dictionary was constructed by collecting SNS hashtags derived from each emotion seed. We also want to explore the priority of each Hangul emotion index. Design/methodology/approach In the process of transforming the matrix through the vector process of words constituting the sentence, weights were extracted using TF-IDF (Term Frequency Inverse Document Frequency), and the dimension reduction technique of the matrix in the emotion set was NMF (Nonnegative Matrix Factorization) algorithm. The emotional dimension was solved by using the characteristic value of the emotional word. The cosine distance algorithm was used to measure the distance between vectors by measuring the similarity of emotion words in the emotion set. Findings Customer needs analysis is a force to read changes in emotions, and Korean emotion word research is the customer's needs. In addition, the ranking of the emotion words within the emotion set will be a special criterion for reading the depth of the emotion. The sentiment index study of this research believes that by providing companies with effective information for emotional marketing, new business opportunities will be expanded and valued. In addition, if the emotion dictionary is eventually connected to the emotional DNA of the product, it will be possible to define the "emotional DNA", which is a set of emotions that the product should have.

Event Sentence Extraction for Online Trend Analysis (온라인 동향 분석을 위한 이벤트 문장 추출 방안)

  • Yun, Bo-Hyun
    • The Journal of the Korea Contents Association
    • /
    • v.12 no.9
    • /
    • pp.9-15
    • /
    • 2012
  • A conventional event sentence extraction research doesn't learn the 3W features in the learning step and applies the rule on whether the 3W feature exists in the extraction step. This paper presents a sentence weight based event sentence extraction method that calculates the weight of the 3W features in the learning step and applies the weight of the 3W features in the extraction step. In the experimental result, we show that top 30% features by the $TF{\times}IDF$ weighting method is good in the feature filtering. In the real estate domain of the public issue, the performance of sentence weight based event sentence extraction method is improved by who and when of 3W features. Moreover, In the real estate domain of the public issue, the sentence weight based event sentence extraction method is better than the other machine learning based extraction method.

A Study on the Changes of the Restaurant Industry Before and After COVID-19 Using BigData (빅데이터를 활용한 코로나 19 이전과 이후 외식산업의 변화에 관한 연구)

  • Ahn, Youn Ju
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.6
    • /
    • pp.787-793
    • /
    • 2022
  • After COVID-19, with the emergence of social distancing, non-face-to-face services, and home economics, visiting dining out is rapidly being replaced by non-face-to-face dining out. The purpose of this study is to find ways to create a safe dining culture centered on living quarantine in line with the changing trend of the restaurant industry after the outbreak of COVID-19, establish the direction of food culture improvement projects, and enhance the effectiveness of the project. This study used TEXTOM to collect and refine search frequency, perform TF-IDF analysis, and Ucinet6 programs to implement visualization using NetDraw from January 1, 2018 to October 31, 2019 and December 31, 2021, and identified the network between nodes of key keywords. Finally, clustering between them was performed through Concor analysis. As a result of the study, if you check the frequency of searches before and after COVID-19, it can be seen that the COVID-19 pandemic greatly affects the changes in the restaurant industry.

A Study on the Finding of Promising Export Items in Defense industry for Export Market Expansion-Focusing on Text Mining Analysis-

  • Yeo, Seoyoon;Jeong, Jong Hee;Kim, Seong Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.10
    • /
    • pp.235-243
    • /
    • 2022
  • This paper aims to find promising export items for market expansion of defense export items. Germany, the UK, and France were selected as export target countries to obtain unstructured forecast data on weapons system acquisition plans for the next ten years by each country. Using the TF-IDF in text mining analysis, keywords that appeared frequently in data from three countries were derived. As a result of this paper, keywords for each country's major acquisition projects drawing. However, most of the derived keywords were related to mainstay weapon systems produced by domestic defense companies in each country. To discover promising export items from text mining, we proposed that the drawn keywords are distinguished as similar weapon systems. In addition, we assort the weapon systems that the three countries will get a plan to acquire commonly. As a result of this paper, it can be seen that the current promising export item is a weapon system related to the information system. Prioritizing overseas demands using key words can set clear market entry goals. In the case of domestic companies based on needs, it is possible to establish a specific entry strategy. Relevant organizations also can provide customized marketing support.

A Study on the Archival Information Services of Economic Policy Using Text Mining Methods: Focusing on Economic Policy Directions (텍스트 마이닝을 활용한 경제정책기록서비스 연구: 경제정책방향을 중심으로)

  • Yeon, Jihyun;Kim, Sungwon
    • Journal of Korean Society of Archives and Records Management
    • /
    • v.22 no.2
    • /
    • pp.117-133
    • /
    • 2022
  • The archival content listed arbitrarily makes it difficult for users to efficiently access the records of major economic policies, especially given that they use it without understanding the required period and context. Using the text mining techniques in the 30-year economic policy direction from 1991 to 2021, this paper derives economic-related keywords and changes that the government mainly dealt with. It collects and preprocesses major economic policies' background, main content, and body text and conducts text frequency, term frequency-inverse document frequency (TF-IDF), network, and time series analyses. Based on these analyses, the following words are recorded in order of frequency: "job(일자리)," "competitive(경쟁력)," and "restructuring(구조조정)." In addition, the relative ratio of "job (일자리)," "real estate(부동산)," and "corporation(기업)," by year was analyzed in terms of chronological order while presenting major keywords mentioned by each government. Based on the results, this study presents implications for developing and broadening the area of archival information services related to economic policies.

Media-based Analysis of Gasoline Inventory with Korean Text Summarization (한국어 문서 요약 기법을 활용한 휘발유 재고량에 대한 미디어 분석)

  • Sungyeon Yoon;Minseo Park
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.5
    • /
    • pp.509-515
    • /
    • 2023
  • Despite the continued development of alternative energies, fuel consumption is increasing. In particular, the price of gasoline fluctuates greatly according to fluctuations in international oil prices. Gas stations adjust their gasoline inventory to respond to gasoline price fluctuations. In this study, news datasets is used to analyze the gasoline consumption patterns through fluctuations of the gasoline inventory. First, collecting news datasets with web crawling. Second, summarizing news datasets using KoBART, which summarizes the Korean text datasets. Finally, preprocessing and deriving the fluctuations factors through N-Gram Language Model and TF-IDF. Through this study, it is possible to analyze and predict gasoline consumption patterns.

Building and Analyzing Panic Disorder Social Media Corpus for Automatic Deep Learning Classification Model (딥러닝 자동 분류 모델을 위한 공황장애 소셜미디어 코퍼스 구축 및 분석)

  • Lee, Soobin;Kim, Seongdeok;Lee, Juhee;Ko, Youngsoo;Song, Min
    • Journal of the Korean Society for information Management
    • /
    • v.38 no.2
    • /
    • pp.153-172
    • /
    • 2021
  • This study is to create a deep learning based classification model to examine the characteristics of panic disorder and to classify the panic disorder tendency literature by the panic disorder corpus constructed for the present study. For this purpose, 5,884 documents of the panic disorder corpus collected from social media were directly annotated based on the mental disease diagnosis manual and were classified into panic disorder-prone and non-panic-disorder documents. Then, TF-IDF scores were calculated and word co-occurrence analysis was performed to analyze the lexical characteristics of the corpus. In addition, the co-occurrence between the symptom frequency measurement and the annotated symptom was calculated to analyze the characteristics of panic disorder symptoms and the relationship between symptoms. We also conducted the performance evaluation for a deep learning based classification model. Three pre-trained models, BERT multi-lingual, KoBERT, and KcBERT, were adopted for classification model, and KcBERT showed the best performance among them. This study demonstrated that it can help early diagnosis and treatment of people suffering from related symptoms by examining the characteristics of panic disorder and expand the field of mental illness research to social media.

Properties of the Twenty-seven Pulses in DongUiBoGam Based on the Eight Important Pulses (팔요맥을 중심으로 살펴본 『동의보감』 27맥 속성 연구)

  • Lee, Taehyung;Jung, Won-Mo;Go, Byeongho;Park, Hi-Joon;Kim, Namil;Chae, Younbyoung
    • Korean Journal of Acupuncture
    • /
    • v.32 no.4
    • /
    • pp.151-159
    • /
    • 2015
  • Objectives : Pulse diagnosis is considered particularly important among several methods of diagnosis in DongUiBoGam. In spite of its importance, numerous and various pulse descriptions made it difficult to learn and practice pulse diagnosis. In this article, we tried to analyze properties of the twenty-seven pulses from pulse diagnosis cases from DongUiBoGam to enable the practical understanding of pulse diagnosis. Methods : We constituted the four axis according to the eight important pulses. And we analyzed properties of the twenty-seven pulses through the relationship between the four pairs of important pulses and the twenty-seven pulses. To quantify the relevances of important pulses to the twenty-seven pulses, we used the term frequency-inverse document frequency(TF-IDF) method. Results : We could elicit properties of the twenty-seven pulses according to the four axis. Also, we reexamined the categorization of the seven exterior pulses / the eight interior pulses and the similar pulses from DongUiBoGam with the analysis results. Conclusions : We could understand properties of the twenty-seven pulses more specifically with the eight important pulses. And we also could see the relationship among the twenty-seven pulses on each axis. However, the limitation arising from insufficient number of pulse diagnosis cases in this research requires further research with more sources such as other traditional medical records or clinical records in the present time.