• Title/Summary/Keyword: Big data collection

Search Result 348, Processing Time 0.034 seconds

Development of Cloud based Data Collection and Analysis for Manufacturing (클라우드 기반의 생산설비 데이터 수집 및 분석 시스템 개발)

  • Young-Dong Lee
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.23 no.4
    • /
    • pp.216-221
    • /
    • 2022
  • The 4th industrial revolution is accelerating the transition to digital innovation in various aspects of our daily lives, and efforts for manufacturing innovation are continuing in the manufacturing industry, such as smart factories. The 4th industrial revolution technology in manufacturing can be used based on AI, big data, IoT, cloud, and robots. Through this, it is required to develop a technology to establish a production facility data collection and analysis system that has evolved from the existing automation and to find the cause of defects and minimize the defect rate. In this paper, we implemented a system that collects power, environment, and status data from production facility sites through IoT devices, quantifies them in real-time in a cloud computing environment, and displays them in the form of MQTT-based real-time infographics using widgets. The real-time sensor data transmitted from the IoT device is stored to the cloud server through a Rest API method. In addition, the administrator could remotely monitor the data on the dashboard and analyze it hourly and daily.

Deep Web and MapReduce

  • Tao, Yufei
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.3
    • /
    • pp.147-158
    • /
    • 2013
  • This invited paper introduces results on Web science and technology obtained during work with the Korea Advanced Institute of Science and Technology. In the first part, we discuss algorithms for exploring the deep Web, which refers to the collection of Web pages that cannot be reached by conventional Web crawlers. In the second part, we discuss sorting algorithms on the MapReduce system, which has become a dominant paradigm for massive parallel computing.

A Public Open Civil Complaint Data Analysis Model to Improve Spatial Welfare for Residents - A Case Study of Community Welfare Analysis in Gangdong District - (거주민 공간복지 향상을 위한 공공 개방 민원 데이터 분석 모델 - 강동구 공간복지 분석 사례를 중심으로 -)

  • Shin, Dongyoun
    • Journal of KIBIM
    • /
    • v.13 no.3
    • /
    • pp.39-47
    • /
    • 2023
  • This study aims to introduce a model for enhancing community well-being through the utilization of public open data. To objectively assess abstract notions of residential satisfaction, text data from complaints is analyzed. By leveraging accessible public data, costs related to data collection are minimized. Initially, relevant text data containing civic complaints is collected and refined by removing extraneous information. This processed data is then combined with meaningful datasets and subjected to topic modeling, a text mining technique. The insights derived are visualized using Geographic Information System (GIS) and Application Programming Interface (API) data. The efficacy of this analytical model was demonstrated in the Godeok/Gangil area. The proposed methodology allows for comprehensive analysis across time, space, and categories. This flexible approach involves incorporating specific public open data as needed, all within the overarching framework.

A Study on the Analysis of Aviation Safety Data Structure and Standard Classification (항공안전데이터 구조 분석 및 표준 분류체계에 관한 연구)

  • Kim, Jun Hwan;Lim, Jae Jin;Lee, Jang Ryong
    • Journal of the Korean Society for Aviation and Aeronautics
    • /
    • v.28 no.4
    • /
    • pp.89-101
    • /
    • 2020
  • In order to enhance the safety of the international aviation industry, the International Civil Aviation Organization has recommended establishing an operational foundation for systematic and integrated collection, storage, analysis and sharing of aviation safety data. Accordingly, the Korea aviation industry also needs to comprehensively manage the safety data which generated and collected by various stakeholders related to aviation safety, and through this, it is necessary to previously identify and remove hazards that may cause accident. For more effective data management and utilization, a standard structure should be established to enable integrated management and sharing of safety data. Therefore, this study aims to propose the framework about how to manage and integrate the aviation safety data for big data-based aviation safety management and shared platform.

Analysis of Public Perception and Policy Implications of Foreign Workers through Social Big Data analysis (소셜 빅데이터분석을 통한 외국인근로자에 관한 국민 인식 분석과 정책적 함의)

  • Ha, Jae-Been;Lee, Do-Eun
    • Journal of Digital Convergence
    • /
    • v.19 no.11
    • /
    • pp.1-10
    • /
    • 2021
  • This paper aimed to look at the awareness of foreign workers in social platforms by using text mining, one of the big data techniques and draw suggestions for foreign workers. To achieve this purpose, data collection was conducted with search keyword 'Foreign Worker' from Jan. 1, to Dec. 31, 2020, and frequency analysis, TF-IDF analysis, and degree centrality analysis and 100 parent keywords were drawn for comparison. Furthermore, Ucinet6.0 and Netdraw were used to analyze semantic networks, and through CONCOR analysis, data were clustered into the following eight groups: foreigner policy issue, regional community issue, business owner's perspective issue, employment issue, working environment issue, legal issue, immigration issue, and human rights issue. Based on such analyzed results, it identified national awareness of foreign workers and main issues and provided the basic data on policy proposals for foreign workers and related researches.

Development of big data based Skin Care Information System SCIS for skin condition diagnosis and management

  • Kim, Hyung-Hoon;Cho, Jeong-Ran
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.3
    • /
    • pp.137-147
    • /
    • 2022
  • Diagnosis and management of skin condition is a very basic and important function in performing its role for workers in the beauty industry and cosmetics industry. For accurate skin condition diagnosis and management, it is necessary to understand the skin condition and needs of customers. In this paper, we developed SCIS, a big data-based skin care information system that supports skin condition diagnosis and management using social media big data for skin condition diagnosis and management. By using the developed system, it is possible to analyze and extract core information for skin condition diagnosis and management based on text information. The skin care information system SCIS developed in this paper consists of big data collection stage, text preprocessing stage, image preprocessing stage, and text word analysis stage. SCIS collected big data necessary for skin diagnosis and management, and extracted key words and topics from text information through simple frequency analysis, relative frequency analysis, co-occurrence analysis, and correlation analysis of key words. In addition, by analyzing the extracted key words and information and performing various visualization processes such as scatter plot, NetworkX, t-SNE, and clustering, it can be used efficiently in diagnosing and managing skin conditions.

Face Morphing Using Generative Adversarial Networks (Generative Adversarial Networks를 이용한 Face Morphing 기법 연구)

  • Han, Yoon;Kim, Hyoung Joong
    • Journal of Digital Contents Society
    • /
    • v.19 no.3
    • /
    • pp.435-443
    • /
    • 2018
  • Recently, with the explosive development of computing power, various methods such as RNN and CNN have been proposed under the name of Deep Learning, which solve many problems of Computer Vision have. The Generative Adversarial Network, released in 2014, showed that the problem of computer vision can be sufficiently solved in unsupervised learning, and the generation domain can also be studied using learned generators. GAN is being developed in various forms in combination with various models. Machine learning has difficulty in collecting data. If it is too large, it is difficult to refine the effective data set by removing the noise. If it is too small, the small difference becomes too big noise, and learning is not easy. In this paper, we apply a deep CNN model for extracting facial region in image frame to GAN model as a preprocessing filter, and propose a method to produce composite images of various facial expressions by stably learning with limited collection data of two persons.

An Analysis of Changes in Perception of Metaverse through Big Data - Comparing Before and After COVID-19 - (빅데이터 분석을 통한 메타버스에 대한 인식 변화 분석 - 코로나19 발생 전후 비교를 중심으로 -)

  • Kang, Yu Rim;Kim, Mun Young
    • Fashion & Textile Research Journal
    • /
    • v.24 no.5
    • /
    • pp.593-604
    • /
    • 2022
  • The purpose of this study is to analyze the flow of change in perception of metaverse before and after COVID-19 through big data analysis. This research method used Textom to collect all data, including metaverse for two years before COVID-19 (2018.1.1~2019.11.30) and after COVID-19 outbreak (2020.1.11~2021.12.31), and the collection channels were selected by Naver and Google. The collected data were text mining, and word frequency, TF-IDF, word cloud, network analysis, and emotional analysis were conducted. As a result of the analysis, first, hotels, weddings, and glades were commonly extracted as social issues related to metaverse before and after COVID-19, and keywords such as robots and launches were derived, so the frequency of keywords related to hotels and weddings was high. Second, the association of the pre-COVID-19 metaverse keywords was platform-oriented, content-oriented, economic-oriented, and online promotion-oriented, and post-COVID-19 clusters were event-oriented, ontact sales-oriented, stock-oriented, and new businesses. Third, positive keywords such as likes, interest, and joy before COVID-19 were high, and positive keywords such as likes, joy, and interest after COVID-19. In conclusion, through this study, it was found that metaverse has firmly established itself as a new platform business model that can be used in various fields such as tourism, travel, festivals, and education using smart technology and metaverse.

Research Progress and Development of Technology in Tourism Research: A Bibliometric Analysis

  • Zhong, Lina;Zhu, Mengyao;Sun, Sunny;Law, Rob
    • Journal of Smart Tourism
    • /
    • v.1 no.2
    • /
    • pp.3-12
    • /
    • 2021
  • The interaction between technology and tourism has been a dynamic research area recently. This study aims to review the progress and development of technology in tourism research via a bibliometric analysis. We derive the source data from the Web of Science (WoS) core collection and use CiteSpace for bibliometric analysis, including countries, institutions, authors, categories, references, and keywords. The analysis results are as follows: i) The number of published articles on the role of technology in tourism has increased in recent years. ii) Technology-related articles in tourism are abundant in Tourism Management, Journal of Travel Research, and Annals of Tourism Research. iii) The countries with the most contributions are China, the US, and the UK. The most active institutions are the Hong Kong Polytechnic University, University of Central Florida, Bournemouth University, University of Queensland, and Kyung Hee University. iv) The reference analysis results identify eight extensively researched topics from the most cited papers, and the keyword burst analysis results present an emerging trend. This study identifies the effect and development of technology in tourism research. Our findings provide implications for researchers about the current research focus of technology and the future research trend of technology in the tourism field.

A Study on the Changes in Perspectives on Unwed Mothers in S.Korea and the Direction of Government Polices: 1995~2020 Social Media Big Data Analysis (한국미혼모에 대한 관점 변화와 정부정책의 방향: 1995년~2020년 소셜미디어 빅데이터 분석)

  • Seo, Donghee;Jun, Boksun
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.12
    • /
    • pp.305-313
    • /
    • 2021
  • This study collected and analyzed big data from 1995 to 2020, focusing on the keywords "unwed mother", "single mother," and "single mom" to present appropriate government support policy directions according to changes in perspectives on unwed mothers. Big data collection platform Textom was used to collect data from portal search sites Naver and Daum and refine data. The final refined data were word frequency analysis, TF-IDF analysis, an N-gram analysis provided by Textom. In addition, Network analysis and CONCOR analysis were conducted through the UCINET6 program. As a result of the study, similar words appeared in word frequency analysis and TF-IDF analysis, but they differed by year. In the N-gram analysis, there were similarities in word appearance, but there were many differences in frequency and form of words appearing in series. As a result of CONCOR analysis, it was found that different clusters were formed by year. This study confirms the change in the perspective of unwed mothers through big data analysis, suggests the need for unwed mothers policies for various options for independent women, and policies that embrace pregnancy, childbirth, and parenting without discrimination within the new family form.