• Title/Summary/Keyword: news big data

Search Result 287, Processing Time 0.025 seconds

Analysis of Twitter for 2012 South Korea Presidential Election by Text Mining Techniques (텍스트 마이닝을 이용한 2012년 한국대선 관련 트위터 분석)

  • Bae, Jung-Hwan;Son, Ji-Eun;Song, Min
    • Journal of Intelligence and Information Systems
    • /
    • v.19 no.3
    • /
    • pp.141-156
    • /
    • 2013
  • Social media is a representative form of the Web 2.0 that shapes the change of a user's information behavior by allowing users to produce their own contents without any expert skills. In particular, as a new communication medium, it has a profound impact on the social change by enabling users to communicate with the masses and acquaintances their opinions and thoughts. Social media data plays a significant role in an emerging Big Data arena. A variety of research areas such as social network analysis, opinion mining, and so on, therefore, have paid attention to discover meaningful information from vast amounts of data buried in social media. Social media has recently become main foci to the field of Information Retrieval and Text Mining because not only it produces massive unstructured textual data in real-time but also it serves as an influential channel for opinion leading. But most of the previous studies have adopted broad-brush and limited approaches. These approaches have made it difficult to find and analyze new information. To overcome these limitations, we developed a real-time Twitter trend mining system to capture the trend in real-time processing big stream datasets of Twitter. The system offers the functions of term co-occurrence retrieval, visualization of Twitter users by query, similarity calculation between two users, topic modeling to keep track of changes of topical trend, and mention-based user network analysis. In addition, we conducted a case study on the 2012 Korean presidential election. We collected 1,737,969 tweets which contain candidates' name and election on Twitter in Korea (http://www.twitter.com/) for one month in 2012 (October 1 to October 31). The case study shows that the system provides useful information and detects the trend of society effectively. The system also retrieves the list of terms co-occurred by given query terms. We compare the results of term co-occurrence retrieval by giving influential candidates' name, 'Geun Hae Park', 'Jae In Moon', and 'Chul Su Ahn' as query terms. General terms which are related to presidential election such as 'Presidential Election', 'Proclamation in Support', Public opinion poll' appear frequently. Also the results show specific terms that differentiate each candidate's feature such as 'Park Jung Hee' and 'Yuk Young Su' from the query 'Guen Hae Park', 'a single candidacy agreement' and 'Time of voting extension' from the query 'Jae In Moon' and 'a single candidacy agreement' and 'down contract' from the query 'Chul Su Ahn'. Our system not only extracts 10 topics along with related terms but also shows topics' dynamic changes over time by employing the multinomial Latent Dirichlet Allocation technique. Each topic can show one of two types of patterns-Rising tendency and Falling tendencydepending on the change of the probability distribution. To determine the relationship between topic trends in Twitter and social issues in the real world, we compare topic trends with related news articles. We are able to identify that Twitter can track the issue faster than the other media, newspapers. The user network in Twitter is different from those of other social media because of distinctive characteristics of making relationships in Twitter. Twitter users can make their relationships by exchanging mentions. We visualize and analyze mention based networks of 136,754 users. We put three candidates' name as query terms-Geun Hae Park', 'Jae In Moon', and 'Chul Su Ahn'. The results show that Twitter users mention all candidates' name regardless of their political tendencies. This case study discloses that Twitter could be an effective tool to detect and predict dynamic changes of social issues, and mention-based user networks could show different aspects of user behavior as a unique network that is uniquely found in Twitter.

A Semantic Text Model with Wikipedia-based Concept Space (위키피디어 기반 개념 공간을 가지는 시멘틱 텍스트 모델)

  • Kim, Han-Joon;Chang, Jae-Young
    • The Journal of Society for e-Business Studies
    • /
    • v.19 no.3
    • /
    • pp.107-123
    • /
    • 2014
  • Current text mining techniques suffer from the problem that the conventional text representation models cannot express the semantic or conceptual information for the textual documents written with natural languages. The conventional text models represent the textual documents as bag of words, which include vector space model, Boolean model, statistical model, and tensor space model. These models express documents only with the term literals for indexing and the frequency-based weights for their corresponding terms; that is, they ignore semantical information, sequential order information, and structural information of terms. Most of the text mining techniques have been developed assuming that the given documents are represented as 'bag-of-words' based text models. However, currently, confronting the big data era, a new paradigm of text representation model is required which can analyse huge amounts of textual documents more precisely. Our text model regards the 'concept' as an independent space equated with the 'term' and 'document' spaces used in the vector space model, and it expresses the relatedness among the three spaces. To develop the concept space, we use Wikipedia data, each of which defines a single concept. Consequently, a document collection is represented as a 3-order tensor with semantic information, and then the proposed model is called text cuboid model in our paper. Through experiments using the popular 20NewsGroup document corpus, we prove the superiority of the proposed text model in terms of document clustering and concept clustering.

A Study on the Information Strategy Planing for the Construction of the Online Information System for the Transaction of Art (미술품 거래정보 온라인 제공시스템 구축을 위한 정보전략계획)

  • Seo, Byeong-Min
    • Journal of Digital Convergence
    • /
    • v.17 no.11
    • /
    • pp.61-70
    • /
    • 2019
  • The The government has recently announced its mid- to long-term plans for promoting art. With the advent of the 4th industrial revolution, contemporary art contents that are integrated with Intelligent Information Technologies such as Artificial Intelligence (AI), Virtual Reality (VR), and Big Data are being introduced, and social interest in humanities and creative convergence is rising. In addition, the industrialization of the art market is expanding amid the rising popularity of art among the general public and the growing interest of art as an investment replacement system, along with the strengthening of the creative personality education of our Education Ministry. Therefore, it is necessary to establish a strategy for transparency and revitalization of the art market by providing comprehensive information such as search functions, analysis data, and criticism by writer and price. This paper has established an information system plan for the establishment of an online supply system for art transaction information, providing auction transaction information for art market, providing report and news for art market, providing public relations platform, and providing art market analysis service and membership relationship management service. To this end, the future model was established through environmental analysis and focus analysis of the art market, and strategic tasks and implementation plans were established accordingly.

A Study on the Analysis of Related Information through the Establishment of the National Core Technology Network: Focused on Display Technology (국가핵심기술 관계망 구축을 통한 연관정보 분석연구: 디스플레이 기술을 중심으로)

  • Pak, Se Hee;Yoon, Won Seok;Chang, Hang Bae
    • The Journal of Society for e-Business Studies
    • /
    • v.26 no.2
    • /
    • pp.123-141
    • /
    • 2021
  • As the dependence of technology on the economic structure increases, the importance of National Core Technology is increasing. However, due to the nature of the technology itself, it is difficult to determine the scope of the technology to be protected because the scope of the relation is abstract and information disclosure is limited due to the nature of the National Core Technology. To solve this problem, we propose the most appropriate literature type and method of analysis to distinguish important technologies related to National Core Technology. We conducted a pilot test to apply TF-IDF, and LDA topic modeling, two techniques of text mining analysis for big data analysis, to four types of literature (news, papers, reports, patents) collected with National Core Technology keywords in the field of Display industry. As a result, applying LDA theme modeling to patent data are highly relevant to National Core Technology. Important technologies related to the front and rear industries of displays, including OLEDs and microLEDs, were identified, and the results were visualized as networks to clarify the scope of important technologies associated with National Core Technology. Throughout this study, we have clarified the ambiguity of the scope of association of technologies and overcome the limited information disclosure characteristics of national core technologies.

Effects of Exposure to Cooking Show Contents on the Consumption of Agricultural Products: Focused on Potato Consumption (쿡방 콘텐츠 노출이 농식품 소비에 미치는 효과: 감자 소비를 중심으로)

  • Rah, HyungChul;Kim, Hyeon-Woong;Ko, Hyeonseok;Shin, Jaehoon;Cho, Yongbeen;Nasridinov, Aziz;Yoo, Kwan-Hee
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.12
    • /
    • pp.400-407
    • /
    • 2021
  • Recently, mukbang and cookbang or cooking shows on TV and YouTube channels have increased, and the influences of these broadcasts on food consumption have been gradually increasing. There were several news articles on 'Baek Jong-won effect', in which the consumption of the agri-food Mr. Jong-won Baek mentioned on his broadcast soared, and even foods named after him are on the market. In this study, Mr. Jong-won Baek, who produces influential cooking contents through various media, was taken as a representative example. We evaluated if 'Baek Jong-won effect' exists on potato consumption, which Mr. Jong-won Baek broadcasted potato cooking recipes on TV and YouTube. After the potato recipe was broadcasted for the first time on the TV show called HomeFoodRescue, the differences in the amount of money to purchase potatoes before and after the broadcast were estimated by using the money amount to purchase data of Agri-food consumers panel and the difference-in-differences method at 6 time points (3, 6, 9, 12, 24, and 36 months). Among the time points analyzed, the potato purchases at post-broadcast were less than those at pre-broadcast. No results were observed suggesting the existence of 'Baek Jong-won effect' on potato consumption through HomeFoodRescue show in the study.

A Study of 'Emotion Trigger' by Text Mining Techniques (텍스트 마이닝을 이용한 감정 유발 요인 'Emotion Trigger'에 관한 연구)

  • An, Juyoung;Bae, Junghwan;Han, Namgi;Song, Min
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.2
    • /
    • pp.69-92
    • /
    • 2015
  • The explosion of social media data has led to apply text-mining techniques to analyze big social media data in a more rigorous manner. Even if social media text analysis algorithms were improved, previous approaches to social media text analysis have some limitations. In the field of sentiment analysis of social media written in Korean, there are two typical approaches. One is the linguistic approach using machine learning, which is the most common approach. Some studies have been conducted by adding grammatical factors to feature sets for training classification model. The other approach adopts the semantic analysis method to sentiment analysis, but this approach is mainly applied to English texts. To overcome these limitations, this study applies the Word2Vec algorithm which is an extension of the neural network algorithms to deal with more extensive semantic features that were underestimated in existing sentiment analysis. The result from adopting the Word2Vec algorithm is compared to the result from co-occurrence analysis to identify the difference between two approaches. The results show that the distribution related word extracted by Word2Vec algorithm in that the words represent some emotion about the keyword used are three times more than extracted by co-occurrence analysis. The reason of the difference between two results comes from Word2Vec's semantic features vectorization. Therefore, it is possible to say that Word2Vec algorithm is able to catch the hidden related words which have not been found in traditional analysis. In addition, Part Of Speech (POS) tagging for Korean is used to detect adjective as "emotional word" in Korean. In addition, the emotion words extracted from the text are converted into word vector by the Word2Vec algorithm to find related words. Among these related words, noun words are selected because each word of them would have causal relationship with "emotional word" in the sentence. The process of extracting these trigger factor of emotional word is named "Emotion Trigger" in this study. As a case study, the datasets used in the study are collected by searching using three keywords: professor, prosecutor, and doctor in that these keywords contain rich public emotion and opinion. Advanced data collecting was conducted to select secondary keywords for data gathering. The secondary keywords for each keyword used to gather the data to be used in actual analysis are followed: Professor (sexual assault, misappropriation of research money, recruitment irregularities, polifessor), Doctor (Shin hae-chul sky hospital, drinking and plastic surgery, rebate) Prosecutor (lewd behavior, sponsor). The size of the text data is about to 100,000(Professor: 25720, Doctor: 35110, Prosecutor: 43225) and the data are gathered from news, blog, and twitter to reflect various level of public emotion into text data analysis. As a visualization method, Gephi (http://gephi.github.io) was used and every program used in text processing and analysis are java coding. The contributions of this study are as follows: First, different approaches for sentiment analysis are integrated to overcome the limitations of existing approaches. Secondly, finding Emotion Trigger can detect the hidden connections to public emotion which existing method cannot detect. Finally, the approach used in this study could be generalized regardless of types of text data. The limitation of this study is that it is hard to say the word extracted by Emotion Trigger processing has significantly causal relationship with emotional word in a sentence. The future study will be conducted to clarify the causal relationship between emotional words and the words extracted by Emotion Trigger by comparing with the relationships manually tagged. Furthermore, the text data used in Emotion Trigger are twitter, so the data have a number of distinct features which we did not deal with in this study. These features will be considered in further study.

Financial Fraud Detection using Text Mining Analysis against Municipal Cybercriminality (지자체 사이버 공간 안전을 위한 금융사기 탐지 텍스트 마이닝 방법)

  • Choi, Sukjae;Lee, Jungwon;Kwon, Ohbyung
    • Journal of Intelligence and Information Systems
    • /
    • v.23 no.3
    • /
    • pp.119-138
    • /
    • 2017
  • Recently, SNS has become an important channel for marketing as well as personal communication. However, cybercrime has also evolved with the development of information and communication technology, and illegal advertising is distributed to SNS in large quantity. As a result, personal information is lost and even monetary damages occur more frequently. In this study, we propose a method to analyze which sentences and documents, which have been sent to the SNS, are related to financial fraud. First of all, as a conceptual framework, we developed a matrix of conceptual characteristics of cybercriminality on SNS and emergency management. We also suggested emergency management process which consists of Pre-Cybercriminality (e.g. risk identification) and Post-Cybercriminality steps. Among those we focused on risk identification in this paper. The main process consists of data collection, preprocessing and analysis. First, we selected two words 'daechul(loan)' and 'sachae(private loan)' as seed words and collected data with this word from SNS such as twitter. The collected data are given to the two researchers to decide whether they are related to the cybercriminality, particularly financial fraud, or not. Then we selected some of them as keywords if the vocabularies are related to the nominals and symbols. With the selected keywords, we searched and collected data from web materials such as twitter, news, blog, and more than 820,000 articles collected. The collected articles were refined through preprocessing and made into learning data. The preprocessing process is divided into performing morphological analysis step, removing stop words step, and selecting valid part-of-speech step. In the morphological analysis step, a complex sentence is transformed into some morpheme units to enable mechanical analysis. In the removing stop words step, non-lexical elements such as numbers, punctuation marks, and double spaces are removed from the text. In the step of selecting valid part-of-speech, only two kinds of nouns and symbols are considered. Since nouns could refer to things, the intent of message is expressed better than the other part-of-speech. Moreover, the more illegal the text is, the more frequently symbols are used. The selected data is given 'legal' or 'illegal'. To make the selected data as learning data through the preprocessing process, it is necessary to classify whether each data is legitimate or not. The processed data is then converted into Corpus type and Document-Term Matrix. Finally, the two types of 'legal' and 'illegal' files were mixed and randomly divided into learning data set and test data set. In this study, we set the learning data as 70% and the test data as 30%. SVM was used as the discrimination algorithm. Since SVM requires gamma and cost values as the main parameters, we set gamma as 0.5 and cost as 10, based on the optimal value function. The cost is set higher than general cases. To show the feasibility of the idea proposed in this paper, we compared the proposed method with MLE (Maximum Likelihood Estimation), Term Frequency, and Collective Intelligence method. Overall accuracy and was used as the metric. As a result, the overall accuracy of the proposed method was 92.41% of illegal loan advertisement and 77.75% of illegal visit sales, which is apparently superior to that of the Term Frequency, MLE, etc. Hence, the result suggests that the proposed method is valid and usable practically. In this paper, we propose a framework for crisis management caused by abnormalities of unstructured data sources such as SNS. We hope this study will contribute to the academia by identifying what to consider when applying the SVM-like discrimination algorithm to text analysis. Moreover, the study will also contribute to the practitioners in the field of brand management and opinion mining.

A Topic Analysis of College Education Using Big Data of News Articles (뉴스 빅데이터를 통해 검토한 대학교육의 토픽 분석)

  • Yang, Ji-Yeon;Koo, Jeong-Ho
    • Journal of Digital Convergence
    • /
    • v.19 no.12
    • /
    • pp.11-20
    • /
    • 2021
  • This study extracts topics related to university education through newspaper articles and analyzes the characteristics of each topic and the reporting patterns of each newspaper. The 9 topics were discovered using LDA. Topic 1 and Topic 3 are related to university support projects for education, but Topic 3 is focused on local universities. Topic 2 is about university education after COVID-19, Topic 4 teaching-learning methods, Topic 5 government policies, Topic 6 the high school education contribution university support projects, Topic 7 the university education vision, Topic 8 internationalization, and Topic 9 the entrance exam. The Chosun Ilbo, Kyunghyang, and Hankyoreh reported a lot of articles associated to lectures after COVID-19, government policies, and comments on university education. Relevant articles since 2016 have been analyzed by newspaper type and before/after COVID-19 through which differences in the topics were studied and discussed. These findings would suggest a basic policy guideline for university education and imply that the positive and negative effects of the media need to be considered.

News Article Analysis of the 4th Industrial Revolution and Advertising before and after COVID-19: Focusing on LDA and Word2vec (코로나 이전과 이후의 4차 산업혁명과 광고의 뉴스기사 분석 : LDA와 Word2vec을 중심으로)

  • Cha, Young-Ran
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.9
    • /
    • pp.149-163
    • /
    • 2021
  • The 4th industrial revolution refers to the next-generation industrial revolution led by information and communication technologies such as artificial intelligence (AI), Internet of Things (IoT), robot technology, drones, autonomous driving and virtual reality (VR) and it also has made a significant impact on the development of the advertising industry. However, the world is rapidly changing to a non-contact, non-face-to-face living environment to prevent the spread of COVID 19. Accordingly, the role of the 4th industrial revolution and advertising is changing. Therefore, in this study, text analysis was performed using Big Kinds to examine the 4th industrial revolution and changes in advertising before and after COVID 19. Comparisons were made between 2019 before COVID 19 and 2020 after COVID 19. Main topics and documents were classified through LDA topic model analysis and Word2vec, a deep learning technique. As the result of the study showed that before COVID 19, policies, contents, AI, etc. appeared, but after COVID 19, the field gradually expanded to finance, advertising, and delivery services utilizing data. Further, education appeared as an important issue. In addition, if the use of advertising related to the 4th industrial revolution technology was mainstream before COVID 19, keywords such as participation, cooperation, and daily necessities, were more actively used for education on advanced technology, while talent cultivation appeared prominently. Thus, these research results are meaningful in suggesting a multifaceted strategy that can be applied theoretically and practically, while suggesting the future direction of advertising in the 4th industrial revolution after COVID 19.

Text Mining Analysis of Media Coverage of Maritime Sports: Perceptions of Yachting, Rowing, and Canoeing (텍스트마이닝을 활용한 해양스포츠에 대한 언론 보도기사 분석: 요트, 조정, 카누를 중심으로)

  • Ji-Hyeon Kim;Bo-Kyeong Kim
    • Journal of the Korean Society of Marine Environment & Safety
    • /
    • v.29 no.6
    • /
    • pp.609-619
    • /
    • 2023
  • This study aimed to investigate the formation of the social perception of domestic maritime sports using text mining analysis of keywords and topics from domestic media coverage over the past 10 years related to representative maritime sports, including yachting, rowing, and canoeing. The results are as follows: First, term frequency (TF) and word cloud analyses identified the top keywords: "maritime," "competition," "experience," "tourism," "world," "yachting," "canoeing," "leisure," and "participation." Second, semantic network analysis revealed that yachting was correlated with terms like "maritime," "industry," "competition," "leisure," "tourism," "boat," "facilities," and "business"; rowing with terms like "competition" and "Chungju"; and canoeing with terms like "maritime," "competition," "experience," "leisure," and "tourism." Third, topic modeling analysis indicated that yachting, rowing, and canoeing are perceived as elite sports and maritime leisure sports. However, the perception of these sports has been demonstrated to have little impact on society, public opinion, and social transformation. In summary, when considering these results comprehensively, it can be concluded that yachting and canoeing have gradually shifted from being perceived as elite sports to essential elements of the maritime leisure industry. Contrariwise, rowing remains primarily associated with elite sports, and its popularization as a maritime leisure sport appears limited at this time.