• The Korean Journal of Applied Statistics
• /
• v.29 no.5
• /
• pp.935-947
• /
• 2016

This paper considers the analysis of patterns in meteorological research and development using a text-mining algorithm as the method of analyzing unstructured data. To analyze text data, we define a list of terms related to meteorological research and development, construct times series of a term-document matrix through data preprocessing, and identify terms that have upward or downward patterns over time. The proposed methodology is applied to multi-year plans funded by Korea Meteorological Administration research and development programs from 2011 to 2015.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2017.05a
• /
• pp.361-363
• /
• 2017

최근 소셜 미디어의 이용률이 폭발적으로 증가함에 따라, 방대한 데이터가 네트워크로 쏟아져 나오고 있다. 이들 데이터는 기존의 정형 데이터뿐만 아니라 이미지, 동영상 등의 비정형 데이터가 있으며, 이들을 포괄하여 빅데이터라고 불린다. 이러한 빅데이터는 오피니언 마이닝, 테스트 마이닝 등의 기술적인 분석 기법과 빅데이터 요약 및 효과적인 표현방법에 대한 시각화 기법에 대하여 활발한 연구가 이루어지고 있다. 이 논문은 인기 있는 사회연결망 서비스인 Twitter의 트윗을 수집하고, 빅데이터 분석 기법인 텍스트 마이닝을 활용하여 2017년 대선에 대하여 분석하였다. 또한 분석된 자료의 효과적인 전달을 위해 워드 클라우드 진행하였다. 이 논문을 위하여 인기 있는 SNS인 Twitter의 최근 7일간 트윗(tweet)을 수집하고 분석하였다.

• The Korean Journal of Applied Statistics
• /
• v.30 no.4
• /
• pp.603-613
• /
• 2017

Many people have recently posted about personal interests on social media. The development of the Internet and computer technology has enabled the storage of digital forms of documents that has resulted in an explosion of the amount of textual data generated; subsequently there is an increased demand for technology to create valuable information from a large number of documents. A text mining technique is often used since text-based data is mostly composed of unstructured forms that are not suitable for the application of statistical analysis or data mining techniques. This study analyzed the Meteorological Yearbook data of the Korea Meteorological Administration (KMA) with a text mining technique. First, a term dictionary was constructed through preprocessing and a term-document matrix was generated. This term dictionary was then used to calculate the annual frequency of term, and observe the change in relative frequency for frequently appearing words. We also used regression analysis to identify terms with increasing and decreasing trends. We analyzed the trends in the Meteorological Yearbook of the KMA and analyzed trends of weather related news, weather status, and status of work trends that the KMA focused on. This study is to provide useful information that can help analyze and improve the meteorological services and reflect meteorological policy.

• Korea Information Processing Society Review
• /
• v.11 no.2
• /
• pp.88-97
• /
• 2004

지난 10년 동안 인터넷의 대중화 덕분으로 World Wide Web과 e-mail은 이미 정보 전달의 일반적인 수단으로 자리를 잡았다. 인터넷과 이에 기반한 e-Busine器는 기존 산업의 전 부분에 걸쳐 효율성과 생산성 증대를 위한 전략적인 도구로 그 중요성이 지속적으로 증대되고 있으며. 지식 노동자들은 업무 시간의 대부분을 문서로 대표되는 정보와 지식을 생산하고 검색하는데 보내고 있다. 새로운 기업정보 자료들이 끊임없이 등록되고, 지난 자료들이 수정, 갱신되는 등 전 세계에 있는 수 많은 기업에서 다양한 지식 자산(Knowledge Asset)들이 지속적으로 생성, 재활용되고 있다. 그러나 이렇게 기업이 생성, 저장, 재 사용하는 정보 중 20% 만이 활용성이 높은 정형 데이터로 구성되어 있고, 나머지 80%는 워드프로세서, e-mail, 프리젠테이션, 스프레드시트, PDF와 같은 복합문서와 인터넷 페이지 등의 비정형 텍스트 형태로 구성되어 있다［1］.(중략)

• Journal of the Korea Institute of Building Construction
• /
• v.24 no.1
• /
• pp.121-131
• /
• 2024

This research explores the potential of leveraging unstructured data from construction supervision documents, which contain detailed inspection insights from independent third-party monitors of building construction processes. With the evolution of analytical methodologies, such unstructured data has been recognized as a valuable source of information, offering diverse insights. The study introduces a framework designed to assess cost performance by applying advanced analytical methods to the unstructured data found in final construction supervision reports. Specifically, key phrases were identified using text mining and social network analysis techniques, and these phrases were then analyzed through binomial logistic regression to assess cost performance. The study found that predictions of cost performance based on unstructured data from supervision documents achieved an accuracy rate of approximately 73%. The findings of this research are anticipated to serve as a foundational resource for analyzing various forms of unstructured data generated within the construction sector in future projects.

• Journal of Korea Society of Industrial Information Systems
• /
• v.27 no.1
• /
• pp.111-123
• /
• 2022

In the era of the Covid19, container shipping rates are surging up. Many studies have attempted to investigate the factors affecting a surge in container shipping rates. However, there is limited literature using text mining techniques for analyzing the underlying causes of the surge. This study aims to identify the factors behind the unprecedented surge in shipping rates using network text analysis and LDA topic modeling. For the analysis, we collected the data and keywords from articles in Lloyd's List during past two years(2020-2021). The results of the text analysis showed that the current surge is mainly due to "US-China trade war", "rising blanking sailings", "port congestion", "container shortage", and "unexpected events such as the Suez canal blockage".

• The Korean Journal of Applied Statistics
• /
• v.37 no.2
• /
• pp.119-138
• /
• 2024

Text indicators are increasingly valuable in economic forecasting, but are often hindered by noise and high dimensionality. This study aims to explore post-processing techniques, specifically noise filtering and dimensionality reduction, to normalize text indicators and enhance their utility through empirical analysis. Predictive target variables for the empirical analysis include monthly leading index cyclical variations, BSI (business survey index) All industry sales performance, BSI All industry sales outlook, as well as quarterly real GDP SA (seasonally adjusted) growth rate and real GDP YoY (year-on-year) growth rate. This study explores the Hodrick and Prescott filter, which is widely used in econometrics for noise filtering, and employs sufficient dimension reduction, a nonparametric dimensionality reduction methodology, in conjunction with unstructured text data. The analysis results reveal that noise filtering of text indicators significantly improves predictive accuracy for both monthly and quarterly variables, particularly when the dataset is large. Moreover, this study demonstrated that applying dimensionality reduction further enhances predictive performance. These findings imply that post-processing techniques, such as noise filtering and dimensionality reduction, are crucial for enhancing the utility of text indicators and can contribute to improving the accuracy of economic forecasts.

• The Journal of the Korea Contents Association
• /
• v.22 no.3
• /
• pp.105-111
• /
• 2022

Multiple-choice survey types have been mainly performed to analyze students' needs for online classes. However, in order to analyze the exact needs of students, unstructured data analysis by answer for essay question is required. Big data is applied in various fields because it is possible to analyze unstructured data. This study aims to investigate and analyze what students want subjects or topics for software lecture in general education that process on non-face-to-face online teaching methods. As for the experimental method, keyword analysis and association analysis of big data were performed with unstructured data by giving a subjective questionnaire to students. By the result, we are able to know the keyword what the students want for software lecture, so it will be an important data for planning and designing software lecture of liberal arts in the future as students can grasp the topics they want to learn.

• The Journal of the Convergence on Culture Technology
• /
• v.10 no.1
• /
• pp.609-615
• /
• 2024

The construction industry's fatality count stands at 402, comprising approximately 46% of total industrial accidents. Notably, construction costs less than 5 billion won account for about 69%, so strengthening safety management at small and medium-sized construction sites is required. In this study, 19,511 accident investigation data were collected using web scraping. Through statistical analysis of the collected structured data and text mining analysis of the unstructured data, accident types and causes of accidents were analyzed by construction costs at sites less than 5 billion won. As a result, it was confirmed that there were differences in accident types and causes depending on the construction costs. It is hoped that the results of this study will be used for customized safety management at small and medium-sized construction sites.

• The Korean Journal of Applied Statistics
• /
• v.32 no.2
• /
• pp.301-317
• /
• 2019

There has been significant research conducted on how to discover various insights through text data using statistical techniques. In this study we analyzed text data produced by the Korean National Police Agency to identify trends in the work by year and compare work characteristics among local authorities by identifying distinctive keywords in documents produced by each local authority. A preprocessing according to the characteristics of each data was conducted and the frequency of words for each document was calculated in order to draw a meaningful conclusion. The simple term frequency shown in the document is difficult to describe the characteristics of the keywords; therefore, the frequency for each term was newly calculated using the term frequency-inverse document frequency weights. The L2 norm normalization technique was used to compare the frequency of words. The analysis can be used as basic data that can be newly for future police work improvement policies and as a method to improve the efficiency of the police service that also help identify a demand for improvements in indoor work.