• Title/Summary/Keyword: Web Text Analysis

Search Result 281, Processing Time 0.025 seconds

Trend Analysis of Korean Economy in the Economic Literature by text mining techniques (텍스트 마이닝 기법을 활용한 한국의 경제연구 동향 분석)

  • Song, Hye-Ji;Park, Kyoung-Soo;Jung, Hye-Eun;Song, Min
    • Proceedings of the Korean Society for Information Management Conference
    • /
    • 2013.08a
    • /
    • pp.47-50
    • /
    • 2013
  • 빅데이터를 활용한 데이터 분석 기법 중 비정형 데이터 분석의 하나인 텍스트 마이닝 기법을 활용하여, 외국 학술지에 나타난 한국의 경제 분야 트렌드를 분석한다. 데이터베이스로 Web of Knowledge의 연구논문을 활용하였으며, 키워드 분석, 네트워크 분석, 토픽모델링 분석을 통해 연구 동향 및 지적구조를 파악하는 데 그 목적이 있다.

  • PDF

Data Mining Research on Maehwado Painting Poetry in the Early Joseon Dynasty

  • Haeyoung Park;Younghoon An
    • Journal of Information Processing Systems
    • /
    • v.19 no.4
    • /
    • pp.474-482
    • /
    • 2023
  • Data mining is a technique for extracting valuable information from vast amounts of data by analyzing statistical and mathematical operations, rules, and relationships. In this study, we employed data mining technology to analyze the data concerning the painting poetry of Maehwado (plum blossom paintings) from the early Joseon Dynasty. The data was extracted from the Hanguk Munjip Chonggan (Korean Literary Collections in Classical Chinese) in the Hanguk Gojeon Jonghap database (Korea Classics DB). Using computer information processing techniques, we carried out web scraping and classification of the painting poetry from the Hanguk Munjip Chonggan. Subsequently, we narrowed down our focus to the painting poetry specifically related to Maehwado in the early Joseon Dynasty. Based on this, refined dataset, we conducted an in-depth analysis and interpretation of the text data at the syllable corpus level. As a result, we found a direct correlation between the corpus statistics for each syllable in Maehwado painting poetry and the symbolic meaning of plum blossoms.

A Study on the Semantic Network Analysis of "Cooking Academy" through the Big Data (빅데이터를 활용한 "조리학원"의 의미연결망 분석에 관한 연구)

  • Lee, Seung-Hoo;Kim, Hak-Seon
    • Culinary science and hospitality research
    • /
    • v.24 no.3
    • /
    • pp.167-176
    • /
    • 2018
  • In this study, Big Data was used to collect the information related to 'Cooking Academy' keywords. After collecting all the data, we calculated the frequency through the text mining and selected the main words for future data analysis. Data collection was conducted from Google Web and News during the period from January 1, 2013 to December 31, 2017. The selected 64 words were analyzed by using UCINET 6.0 program, and the analysis results were visualized with NetDraw in order to present the relationship of main words. As a result, it was found that the most important goal for the students from cooking school is to work as a cook, likewise to have practical classes. In addition, we obtained the result that SNS marketing system that the social sites, such as Facebook, Twitter, and Instagram are actively utilized as a marketing strategy of the institute. Therefore, the results can be helpful in searching for the method of utilizing big data and can bring brand-new ideas for the follow-up studies. In practical terms, it will be remarkable material about the future marketing directions and various programs that are improved by the detailed curriculums through semantic network of cooking school by using big data.

Analysis of Shipping and Logistics News Articles using Topic Modeling (토픽모델링을 활용한 해운물류 뉴스 분석)

  • Hee-Young Yoon;Il-Youp Kwak
    • Korea Trade Review
    • /
    • v.46 no.4
    • /
    • pp.61-76
    • /
    • 2021
  • This study focuses on three logistics-related news (Logistics Newspaper, Korea Shipping Gadget, and Korea Shipping Newspaper) in order to present changes in logistics issues, centering on Corona 19, which has recently had the greatest impact in the world. For data collection, two-year news articles in 2019 and 2020 (title, article, content, date, article classification, article URL) were collected through web crawling (using Python's BeautifulSoup, requests module) on the homepages of three representative logistics-related media companies. As for the data analysis methods, fundamental statistical analysis, Latent Dirichlet Allocation (LDA) for topic modeling, and Scattertext were performed. The analysis results were as follows. First, among the three news media related to logistics, the Korea Shipping Newspaper was carrying out the most active media activities. Second, through topic modeling with LDA, eight logistics-related topics were identified, and keywords and significant issues of each topic were presented. Third, the keywords were visually expressed through Scattertext. This is the first study to present changes in the logistics field, focusing on articles from representative logistics-related media in 2019 and 2020. In particular, 2019 and 2020 can be divided into before and after the outbreak of Corona 19, which has had a great impact not only on the logistics field but also on our lives as a whole. For future work, a multi-faceted approach is required, such as comparative studies of logistics issues between countries or presenting implications based on long-term time-series articles.

Hate Speech Detection Using Modified Principal Component Analysis and Enhanced Convolution Neural Network on Twitter Dataset

  • Majed, Alowaidi
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.1
    • /
    • pp.112-119
    • /
    • 2023
  • Traditionally used for networking computers and communications, the Internet has been evolving from the beginning. Internet is the backbone for many things on the web including social media. The concept of social networking which started in the early 1990s has also been growing with the internet. Social Networking Sites (SNSs) sprung and stayed back to an important element of internet usage mainly due to the services or provisions they allow on the web. Twitter and Facebook have become the primary means by which most individuals keep in touch with others and carry on substantive conversations. These sites allow the posting of photos, videos and support audio and video storage on the sites which can be shared amongst users. Although an attractive option, these provisions have also culminated in issues for these sites like posting offensive material. Though not always, users of SNSs have their share in promoting hate by their words or speeches which is difficult to be curtailed after being uploaded in the media. Hence, this article outlines a process for extracting user reviews from the Twitter corpus in order to identify instances of hate speech. Through the use of MPCA (Modified Principal Component Analysis) and ECNN, we are able to identify instances of hate speech in the text (Enhanced Convolutional Neural Network). With the use of NLP, a fully autonomous system for assessing syntax and meaning can be established (NLP). There is a strong emphasis on pre-processing, feature extraction, and classification. Cleansing the text by removing extra spaces, punctuation, and stop words is what normalization is all about. In the process of extracting features, these features that have already been processed are used. During the feature extraction process, the MPCA algorithm is used. It takes a set of related features and pulls out the ones that tell us the most about the dataset we give itThe proposed categorization method is then put forth as a means of detecting instances of hate speech or abusive language. It is argued that ECNN is superior to other methods for identifying hateful content online. It can take in massive amounts of data and quickly return accurate results, especially for larger datasets. As a result, the proposed MPCA+ECNN algorithm improves not only the F-measure values, but also the accuracy, precision, and recall.

Economic Feasibility Analysis of Nationwide Expansion of Agro-meteorological Early Warning Service for Weather Risk Management in Korea (농업기상재해 조기경보서비스의 전국 확대에 따른 경제적 타당성 분석)

  • Sangtaek Seo;Yun Hee Jeong;Soo Jin Kim;Kyo-Moon Shim
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.25 no.3
    • /
    • pp.236-244
    • /
    • 2023
  • The purpose of this study was to examine the economic feasibility of providing services according to the nationwide expansion of early warning services. The net present value method, one of the cost-benefit analysis methods, was applied to the analysis. As a benefit item that constituted the net present value, the damage reduction amount using crop insurance data and the willingness to pay for the use of early warning services were used. The cost items included system construction and maintenance costs, and text transmission costs. As a result of the analysis, it was found that the nationwide expansion of early warning services had economic feasibility, and its economic effect varied depending on the level of text message use (10 % to 40 %, 10 %p interval) of participating farmers. In the future, the economic effect of early warning services is expected to increase further due to the increase in the number of farmers participating in early warning services and the increase in crop damage caused by climate change. It is necessary to further enhance the economic effect of early warning services by actively utilizing information delivery means through apps or the web as well as text messages.

A Study on Negation Handling and Term Weighting Schemes and Their Effects on Mood-based Text Classification (감정 기반 블로그 문서 분류를 위한 부정어 처리 및 단어 가중치 적용 기법의 효과에 대한 연구)

  • Jung, Yu-Chul;Choi, Yoon-Jung;Myaeng, Sung-Hyon
    • Korean Journal of Cognitive Science
    • /
    • v.19 no.4
    • /
    • pp.477-497
    • /
    • 2008
  • Mood classification of blog text is an interesting problem, with a potential for a variety of services involving the Web. This paper introduces an approach to mood classification enhancements through the normalized negation n-grams which contain mood clues and corpus-specific term weighting(CSTW). We've done experiments on blog texts with two different classification methods: Enhanced Mood Flow Analysis(EMFA) and Support Vector Machine based Mood Classification(SVMMC). It proves that the normalized negation n-gram method is quite effective in dealing with negations and gave gradual improvements in mood classification with EMF A. From the selection of CSTW, we noticed that the appropriate weighting scheme is important for supporting adequate levels of mood classification performance because it outperforms the result of TF*IDF and TF.

  • PDF

Media-based Analysis of Gasoline Inventory with Korean Text Summarization (한국어 문서 요약 기법을 활용한 휘발유 재고량에 대한 미디어 분석)

  • Sungyeon Yoon;Minseo Park
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.5
    • /
    • pp.509-515
    • /
    • 2023
  • Despite the continued development of alternative energies, fuel consumption is increasing. In particular, the price of gasoline fluctuates greatly according to fluctuations in international oil prices. Gas stations adjust their gasoline inventory to respond to gasoline price fluctuations. In this study, news datasets is used to analyze the gasoline consumption patterns through fluctuations of the gasoline inventory. First, collecting news datasets with web crawling. Second, summarizing news datasets using KoBART, which summarizes the Korean text datasets. Finally, preprocessing and deriving the fluctuations factors through N-Gram Language Model and TF-IDF. Through this study, it is possible to analyze and predict gasoline consumption patterns.

Proposal of Research Methodology Using The Measurement of Perception Difference

  • YANG, Hoechang
    • Journal of Wellbeing Management and Applied Psychology
    • /
    • v.2 no.2
    • /
    • pp.39-45
    • /
    • 2019
  • The purpose of this study is to solve the problem of revision or abbreviation of questionnaires based on the previous studies suggested by many existing empirical studies. In addition, this study aims to provide the theoretical basis of the research method which has been variously approached since it presents the methodology that can directly measure the research object. For this purpose, this study proposed a more elaborate analysis method using the differences in perception of individuals who are interested in cognitive research. Specifically, the perception gap(D) can be used as an independent variable, a dependent variable, and a moderating variable. And this study suggested an effective research approach using the measurement of perception difference. The difference of perception suggested that it can be used as a measure to overcome the limitations of existing researches used it as independent variables or mediating variables that measure only one factor of expectation and performance or importance and satisfaction. In addition, it is highly likely that various analyzes on the perception differences, which are the result of measuring target factors for the same person, will be quite effective in the situation where follow-up of respondents is difficult. This study is expected to overcome various limitations reported by empirical studies such as scale utilization problem and follow-up survey difficulty. In future research, it was expected that the limitation of the factor derivation process in the research approach could be complemented by web crawling and text mining of big data analysis.

Analyzing Box-Office Hit Factors Using Big Data: Focusing on Korean Films for the Last 5 Years

  • Hwang, Youngmee;Kim, Kwangsun;Kwon, Ohyoung;Moon, Ilyoung;Shin, Gangho;Ham, Jongho;Park, Jintae
    • Journal of information and communication convergence engineering
    • /
    • v.15 no.4
    • /
    • pp.217-226
    • /
    • 2017
  • Korea has the tenth largest film industry in the world; however, detailed analyses using the factors contributing to successful film commercialization have not been approached. Using big data, this paper analyzed both internal and external factors (including genre, release date, rating, and number of screenings) that contributed to the commercial success of Korea's top 10 ranking films in 2011-2015. The authors developed a WebCrawler to collect text data about each movie, implemented a Hadoop system for data storage, and classified the data using Map Reduce method. The results showed that the characteristic of "release date," followed closely by "rating" and "genre" were the most influential factors of success in the Korean film industry. The analysis in this study is considered groundwork for the development of software that can predict box-office performance.