• Title/Summary/Keyword: Unstructured text data

Search Result 228, Processing Time 0.022 seconds

Big Data Analysis of the Women Who Score Goal Sports Entertainment Program: Focusing on Text Mining and Semantic Network Analysis.

  • Hyun-Myung, Kim;Kyung-Won, Byun
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.15 no.1
    • /
    • pp.222-230
    • /
    • 2023
  • The purpose of this study is to provide basic data on sports entertainment programs by collecting data on unstructured data generated by Naver and Google for SBS entertainment program 'Women Who Score Goal', which began regular broadcast in June 2021, and analyzing public perceptions through data mining, semantic matrix, and CONCOR analysis. Data collection was conducted using Textom, and 27,911 cases of data accumulated for 16 months from June 16, 2021 to October 15, 2022. For the collected data, 80 key keywords related to 'Kick a Goal' were derived through simple frequency and TF-IDF analysis through data mining. Semantic network analysis was conducted to analyze the relationship between the top 80 keywords analyzed through this process. The centrality was derived through the UCINET 6.0 program using NetDraw of UCINET 6.0, understanding the characteristics of the network, and visualizing the connection relationship between keywords to express it clearly. CONCOR analysis was conducted to derive a cluster of words with similar characteristics based on the semantic network. As a result of the analysis, it was analyzed as a 'program' cluster related to the broadcast content of 'Kick a Goal' and a 'Soccer' cluster, a sports event of 'Kick a Goal'. In addition to the scenes about the game of the cast, it was analyzed as an 'Everyday Life' cluster about training and daily life, and a cluster about 'Broadcast Manipulation' that disappointed viewers with manipulation of the game content.

A Study on the Recognition Analysis of Participants in Urban Regeneration Project Using Text Network Analysis Technique (NetMiner): Focused on the Urban Regeneration Leading Area in Suncheon-City

  • Gim, Eo-Jin;Koo, Ja-Hoon
    • International Journal of Advanced Culture Technology
    • /
    • v.7 no.4
    • /
    • pp.246-254
    • /
    • 2019
  • The purpose of this study is to analyze the issues related to urban regeneration project at the present time through surveys and interviews of participants in the urban regeneration leading project of Suncheon city. Most of the comments were related to business fragmentation and things that should be improved in the future. The text network technique is applied to the subject analysis using unstructured text data. As a result of the frequency of appearance and analysis of page rank centrality between words, words of 'parking', 'need', 'lack', 'region' and 'resident' appeared at the top, and the result of analyzing the mediation centrality of key words showed 'culture', 'Need', 'region', 'inflow' and 'lack' appeared at the top. In the network analysis, the most central words appeared, and many words appeared in the important position in the sentence. Text network analysis has provided timely results in terms of sustainability after completion of the Suncheon City Regeneration Leading Project..

Automatic Quality Evaluation with Completeness and Succinctness for Text Summarization (완전성과 간결성을 고려한 텍스트 요약 품질의 자동 평가 기법)

  • Ko, Eunjung;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.2
    • /
    • pp.125-148
    • /
    • 2018
  • Recently, as the demand for big data analysis increases, cases of analyzing unstructured data and using the results are also increasing. Among the various types of unstructured data, text is used as a means of communicating information in almost all fields. In addition, many analysts are interested in the amount of data is very large and relatively easy to collect compared to other unstructured and structured data. Among the various text analysis applications, document classification which classifies documents into predetermined categories, topic modeling which extracts major topics from a large number of documents, sentimental analysis or opinion mining that identifies emotions or opinions contained in texts, and Text Summarization which summarize the main contents from one document or several documents have been actively studied. Especially, the text summarization technique is actively applied in the business through the news summary service, the privacy policy summary service, ect. In addition, much research has been done in academia in accordance with the extraction approach which provides the main elements of the document selectively and the abstraction approach which extracts the elements of the document and composes new sentences by combining them. However, the technique of evaluating the quality of automatically summarized documents has not made much progress compared to the technique of automatic text summarization. Most of existing studies dealing with the quality evaluation of summarization were carried out manual summarization of document, using them as reference documents, and measuring the similarity between the automatic summary and reference document. Specifically, automatic summarization is performed through various techniques from full text, and comparison with reference document, which is an ideal summary document, is performed for measuring the quality of automatic summarization. Reference documents are provided in two major ways, the most common way is manual summarization, in which a person creates an ideal summary by hand. Since this method requires human intervention in the process of preparing the summary, it takes a lot of time and cost to write the summary, and there is a limitation that the evaluation result may be different depending on the subject of the summarizer. Therefore, in order to overcome these limitations, attempts have been made to measure the quality of summary documents without human intervention. On the other hand, as a representative attempt to overcome these limitations, a method has been recently devised to reduce the size of the full text and to measure the similarity of the reduced full text and the automatic summary. In this method, the more frequent term in the full text appears in the summary, the better the quality of the summary. However, since summarization essentially means minimizing a lot of content while minimizing content omissions, it is unreasonable to say that a "good summary" based on only frequency always means a "good summary" in its essential meaning. In order to overcome the limitations of this previous study of summarization evaluation, this study proposes an automatic quality evaluation for text summarization method based on the essential meaning of summarization. Specifically, the concept of succinctness is defined as an element indicating how few duplicated contents among the sentences of the summary, and completeness is defined as an element that indicating how few of the contents are not included in the summary. In this paper, we propose a method for automatic quality evaluation of text summarization based on the concepts of succinctness and completeness. In order to evaluate the practical applicability of the proposed methodology, 29,671 sentences were extracted from TripAdvisor 's hotel reviews, summarized the reviews by each hotel and presented the results of the experiments conducted on evaluation of the quality of summaries in accordance to the proposed methodology. It also provides a way to integrate the completeness and succinctness in the trade-off relationship into the F-Score, and propose a method to perform the optimal summarization by changing the threshold of the sentence similarity.

Research Trends in Record Management Using Unstructured Text Data Analysis (비정형 텍스트 데이터 분석을 활용한 기록관리 분야 연구동향)

  • Deokyong Hong;Junseok Heo
    • Journal of Korean Society of Archives and Records Management
    • /
    • v.23 no.4
    • /
    • pp.73-89
    • /
    • 2023
  • This study aims to analyze the frequency of keywords used in Korean abstracts, which are unstructured text data in the domestic record management research field, using text mining techniques to identify domestic record management research trends through distance analysis between keywords. To this end, 1,157 keywords of 77,578 journals were visualized by extracting 1,157 articles from 7 journal types (28 types) searched by major category (complex study) and middle category (literature informatics) from the institutional statistics (registered site, candidate site) of the Korean Citation Index (KCI). Analysis of t-Distributed Stochastic Neighbor Embedding (t-SNE) and Scattertext using Word2vec was performed. As a result of the analysis, first, it was confirmed that keywords such as "record management" (889 times), "analysis" (888 times), "archive" (742 times), "record" (562 times), and "utilization" (449 times) were treated as significant topics by researchers. Second, Word2vec analysis generated vector representations between keywords, and similarity distances were investigated and visualized using t-SNE and Scattertext. In the visualization results, the research area for record management was divided into two groups, with keywords such as "archiving," "national record management," "standardization," "official documents," and "record management systems" occurring frequently in the first group (past). On the other hand, keywords such as "community," "data," "record information service," "online," and "digital archives" in the second group (current) were garnering substantial focus.

A Study on FIFA Partner Adidas of 2022 Qatar World Cup Using Big Data Analysis

  • Kyung-Won, Byun
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.15 no.1
    • /
    • pp.164-170
    • /
    • 2023
  • The purpose of this study is to analyze the big data of Adidas brand participating in the Qatar World Cup in 2022 as a FIFA partner to understand useful information, semantic connection and context from unstructured data. Therefore, this study collected big data generated during the World Cup from Adidas participating in sponsorship as a FIFA partner for the 2022 Qatar World Cup and collected data from major portal sites to understand its meaning. According to text mining analysis, 'Adidas' was used the most 3,340 times based on the frequency of keyword appearance, followed by 'World Cup', 'Qatar World Cup', 'Soccer', 'Lionel Messi', 'Qatar', 'FIFA', 'Korea', and 'Uniform'. In addition, the TF-IDF rankings were 'Qatar World Cup', 'Soccer', 'Lionel Messi', 'World Cup', 'Uniform', 'Qatar', 'FIFA', 'Ronaldo', 'Korea', and 'Nike'. As a result of semantic network analysis and CONCOR analysis, four groups were formed. First, Cluster A named it 'Qatar World Cup Sponsor' as words such as 'Adidas', 'Nike', 'Qatar World Cup', 'Sponsor', 'Sponsor Company', 'Marketing', 'Nation', 'Launch', 'Official', 'Commemoration' and 'National Team' were formed into groups. Second, B Cluster named it 'Group stage' as words such as 'Qatar', 'Uruguay', 'FIFA' and 'group stage' were formed into groups. Third, C Cluster named it 'Winning' as words such as 'World Cup Winning', 'Champion', 'France', 'Argentina', 'Lionel Messi', 'Advertising' and 'Photograph' formed a group. Fourth, D Cluster named it 'Official Ball' as words such as 'Official Ball', 'World Cup Official Ball', 'Soccer Ball', 'All Times', 'Al Rihla', 'Public', 'Technology' was formed into groups.

Text-mining based Cause Analysis of Accidents at Workplaces in Korea (텍스트 마이닝 기법을 활용한 우리나라 산업재해의 원인분석)

  • Choi, Gi Heung
    • Journal of the Korean Society of Safety
    • /
    • v.37 no.3
    • /
    • pp.9-15
    • /
    • 2022
  • The analysis of the causes of accidents in workplaces where machines and tools are used is essential to improve the effectiveness and efficiency of safety prevention policies in places of employment in Korea. The causes of workplace accidents are not fully understood mainly due to difficulties in analyzing available descriptive information. This study focuses on the automated accident cause analysis in workplaces based on the accident abstracts found in industrial accident reports written in an unstructured descriptive format. The method proposed in this paper is based on text data mining and uses the keyword search function of Excel software to automate the analysis. The analysis results indicate that the primary reason for the frequency of accidents is related to technical aspects at a stage in which dangerous situations occur in the workplace. Accidents due to managerial causes are typically observed when danger exists in the workplace; however, managerial actions play a more important role in reducing accident severity. A small company tends to use unsafe machines and devices, leading to further accidents due to technical causes, whereas managerial causes are more conspicuous as the company grows. To preclude the occurrence of accidents due to inadequate knowledge, the implementation of safety management and the provision of safety education to elderly workers at the early stage of their employment are particularly important for small companies with less than 100 workers.

Analyzing OTT Interactive Content Using Text Mining Method (텍스트 마이닝으로 OTT 인터랙티브 콘텐츠 다시보기)

  • Sukchang Lee
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.5
    • /
    • pp.859-865
    • /
    • 2023
  • In a situation where service providers are increasingly focusing on content development due to the intense competition in the OTT market, interactive content that encourages active participation from viewers is garnering significant attention. In response to this trend, research on interactive content is being conducted more actively. This study aims to analyze interactive content through text mining techniques, with a specific focus on online unstructured data. The analysis includes deriving the characteristics of keywords according to their weight, examining the relationship between OTT platforms and interactive content, and tracking changes in the trends of interactive content based on objective data. To conduct this analysis, detailed techniques such as 'Word Cloud', 'Relationship Analysis', and 'Keyword Trend' are used, and the study also aims to derive meaningful implications from these analyses.

Individual Interests Tracking : Beyond Macro-level Issue Tracking (거시적 이슈 트래킹의 한계 극복을 위한 개인 관심 트래킹 방법론)

  • Liu, Chen;Kim, Namgyu
    • Journal of Information Technology Services
    • /
    • v.13 no.4
    • /
    • pp.275-287
    • /
    • 2014
  • Recently, the volume of unstructured text data generated by various social media has been increasing rapidly; consequently, the use of text mining to support decision-making has also been growing. In particular, academia and industry are paying significant attention to topic analysis in order to discover the main issues from a large volume of text documents. Topic analysis can be regarded as static analysis because it analyzes a snapshot of the distribution of various issues. In contrast, some recent studies have attempted to perform dynamic issue tracking, which analyzes and traces issue trends during a predefined period. However, most traditional issue tracking methods have a common limitation : when a new period is included, topic analysis must be repeated for all the documents of the entire period, rather than being conducted only on the new documents of the added period. Additionally, traditional issue tracking methods do not concentrate on the transition of individuals' interests from certain issues to others, although the methods can illustrate macro-level issue trends. In this paper, we propose an individual interests tracking methodology to overcome the two limitations of traditional issue tracking methods. Our main goal is not to track macro-level issue trends but to analyze trends of individual interests flow. Further, our methodology has extensible characteristics because it analyzes only newly added documents when the period of analysis is extended. In this paper, we also analyze the results of applying our methodology to news articles and their access logs.

Visualization of unstructured personal narratives of perterm birth using text network analysis (텍스트 네트워크 분석을 이용한 조산 경험 이야기의 시각화)

  • Kim, Jeung-Im
    • Women's Health Nursing
    • /
    • v.26 no.3
    • /
    • pp.205-212
    • /
    • 2020
  • Purpose: This study aimed to identify the components of preterm birth (PTB) through women's personal narratives and to visualize clinical symptom expressions (CSEs). Methods: The participants were 11 women who gave birth before 37 weeks of gestational age. Personal narratives were collected by interactive unstructured storytelling via individual interviews, from August 8 to December 4, 2019 after receiving approval of the Institutional Review Board. The textual data were converted to PDF and analyzed using the MAXQDA program (VERBI Software). Results: The participants' mean age was 34.6 (±2.98) years, and five participants had a spontaneous vaginal birth. The following nine components of PTB were identified: obstetric condition, emotional condition, physical condition, medical condition, hospital environment, life-related stress, pregnancy-related stress, spousal support, and informational support. The top three codes were preterm labor, personal characteristics, and premature rupture of membrane, and the codes found for more than half of the participants were short cervix, fear of PTB, concern about fetal well-being, sleep difficulty, insufficient spousal and informational support, and physical difficulties. The top six CSEs were stress, hydramnios, false labor, concern about fetal wellbeing, true labor pain, and uterine contraction. "Stress" was ranked first in terms of frequency and "uterine contraction" had individual attributes. Conclusion: The text network analysis of narratives from women who gave birth preterm yielded nine PTB components and six CSEs. These nine components should be included for developing a reliable and valid scale for PTB risk and stress. The CSEs can be applied for assessing preterm labor, as well as considered as strategies for students in women's health nursing practicum.

A Study on the Quantitative Evaluation of Initial Coin Offering (ICO) Using Unstructured Data (비정형 데이터를 이용한 ICO(Initial Coin Offering) 정량적 평가 방법에 대한 연구)

  • Lee, Han Sol;Ahn, Sangho;Kang, Juyoung
    • Smart Media Journal
    • /
    • v.11 no.5
    • /
    • pp.63-74
    • /
    • 2022
  • Initial public offering (IPO) has a legal framework for investor protection, and because there are various quantitative evaluation factors, objective analysis is possible, and various studies have been conducted. In addition, crowdfunding also has several devices to prevent indiscriminate funding as the legal system for investor protection. On the other hand, the blockchain-based cryptocurrency white paper (ICO), which has recently been in the spotlight, has ambiguous legal means and standards to protect investors and lacks quantitative evaluation methods to evaluate ICOs objectively. Therefore, this study collects online-published ICO white papers to detect fraud in ICOs, performs ICO fraud predictions based on BERT, a text embedding technique, and compares them with existing Random Forest machine learning techniques, and shows the possibility on fraud detection. Finally, this study is expected to contribute to the study of ICO fraud detection based on quantitative methods by presenting the possibility of using a quantitative approach using unstructured data to identify frauds in ICOs.