• Title/Summary/Keyword: large data visualization

Search Result 244, Processing Time 0.034 seconds

Big Data-based Sensor Data Processing and Analysis for IoT Environment (IoT 환경을 위한 빅데이터 기반 센서 데이터 처리 및 분석)

  • Shin, Dong-Jin;Park, Ji-Hun;Kim, Ju-Ho;Kwak, Kwang-Jin;Park, Jeong-Min;Kim, Jeong-Joon
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.19 no.1
    • /
    • pp.117-126
    • /
    • 2019
  • The data generated in the IoT environment is very diverse. Especially, the development of the fourth industrial revolution has made it possible to increase the number of fixed and unstructured data generated in manufacturing facilities such as Smart Factory. With Big Data related solutions, it is possible to collect, store, process, analyze and visualize various large volumes of data quickly and accurately. Therefore, in this paper, we will directly generate data using Raspberry Pi used in IoT environment, and analyze using various Big Data solutions. Collected by using an Sqoop solution collected and stored in the database to the HDFS, and the process is to process the data by using the solutions available Hive parallel processing is associated with Hadoop. Finally, the analysis and visualization of the processed data via the R programming will be used universally to end verification.

A Method for the Extraction of a Subset of Points from a Large Set of Points Affecting the Distribution of Surface Data - A Case Study of Market Area and Competitive Power Analysis by Sales Data of Micro Scale Retail Stores - (평면 데이터 분포에 영향을 끼치는 점 분포의 부분집합 추출 방법 - 소규모 소매점포의 매출자료를 이용한 상권 및 경쟁력 분석기법을 사례로 -)

  • Lee, Jung-Eun;Sadahiro, Yukio
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.9 no.1
    • /
    • pp.1-12
    • /
    • 2006
  • Approaches to spatial analysis differ from the type of spatial objects to be treated. Especially, in here, the case where two spatial data sets coexist is considered. The goal of such case lies on detecting a subset of spatial objects out of a large set that affects the distribution of the other object. However, it is not easy to extract a subset from a large set by visualization just with the help of GIS since huge amount of data are provided nowadays. In this research, therefore, relationship between two different spatial data are analyzed by quantitative measure in the case study of marketing geography. A purchase history data of a small retail store and the location of its competitors are given as source data for the analysis. The goal of analysis from the aspect of this case study is to extract strong competitors of the store that affects the sales amount of the store among many competitors. With the result, therefore, it is expected that market area pattern and competitive power of stores under micro scale retail environment would be understood by quantitative measure.

  • PDF

Hierarchical Browsing Interface for Geo-Referenced Photo Database (위치 정보를 갖는 사진집합의 계층적 탐색 인터페이스)

  • Lee, Seung-Hoon;Lee, Kang-Hoon
    • Journal of the Korea Computer Graphics Society
    • /
    • v.16 no.4
    • /
    • pp.25-33
    • /
    • 2010
  • With the popularization of digital photography, people are now capturing and storing far more photos than ever before. However, the enormous number of photos often discourages the users to identify desired photos. In this paper, we present a novel method for fast and intuitive browsing through large collections of geo-referenced photographs. Given a set of photos, we construct a hierarchical structure of clusters such that each cluster includes a set of spatially adjacent photos and its sub-clusters divide the photo set disjointly. For each cluster, we pre-compute its convex hull and the corresponding polygon area. At run-time, this pre-computed data allows us to efficiently visualize only a fraction of the clusters that are inside the current view and have easily recognizable sizes with respect to the current zoom level. Each cluster is displayed as a single polygon representing its convex hull instead of every photo location included in the cluster. The users can quickly transfer from clusters to clusters by simply selecting any interesting clusters. Our system automatically pans and zooms the view until the currently selected cluster fits precisely into the view with a moderate size. Our user study demonstrates that these new visualization and interaction techniques can significantly improve the capability of navigating over large collections of geo-referenced photos.

Visualizing Unstructured Data using a Big Data Analytical Tool R Language (빅데이터 분석 도구 R 언어를 이용한 비정형 데이터 시각화)

  • Nam, Soo-Tai;Chen, Jinhui;Shin, Seong-Yoon;Jin, Chan-Yong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.05a
    • /
    • pp.151-154
    • /
    • 2021
  • Big data analysis is the process of discovering meaningful new correlations, patterns, and trends in large volumes of data stored in data stores and creating new value. Thus, most big data analysis technology methods include data mining, machine learning, natural language processing, and pattern recognition used in existing statistical computer science. Also, using the R language, a big data tool, we can express analysis results through various visualization functions using pre-processing text data. The data used in this study was analyzed for 21 papers in the March 2021 among the journals of the Korea Institute of Information and Communication Engineering. In the final analysis results, the most frequently mentioned keyword was "Data", which ranked first 305 times. Therefore, based on the results of the analysis, the limitations of the study and theoretical implications are suggested.

  • PDF

Knowledge Mining from Many-valued Triadic Dataset based on Concept Hierarchy (개념계층구조를 기반으로 하는 다치 삼원 데이터집합의 지식 추출)

  • Suk-Hyung Hwang;Young-Ae Jung;Se-Woong Hwang
    • Journal of Platform Technology
    • /
    • v.12 no.3
    • /
    • pp.3-15
    • /
    • 2024
  • Knowledge mining is a research field that applies various techniques such as data modeling, information extraction, analysis, visualization, and result interpretation to find valuable knowledge from diverse large datasets. It plays a crucial role in transforming raw data into useful knowledge across various domains like business, healthcare, and scientific research etc. In this paper, we propose analytical techniques for performing knowledge discovery and data mining from various data by extending the Formal Concept Analysis method. It defines algorithms for representing diverse formats and structures of the data to be analyzed, including models such as many-valued data table data and triadic data table, as well as algorithms for data processing (dyadic scaling and flattening) and the construction of concept hierarchies and the extraction of association rules. The usefulness of the proposed technique is empirically demonstrated by conducting experiments applying the proposed method to public open data.

  • PDF

Development of the KnowledgeMatrix as an Informetric Analysis System (계량정보분석시스템으로서의 KnowledgeMatrix 개발)

  • Lee, Bang-Rae;Yeo, Woon-Dong;Lee, June-Young;Lee, Chang-Hoan;Kwon, Oh-Jin;Moon, Yeong-Ho
    • The Journal of the Korea Contents Association
    • /
    • v.8 no.1
    • /
    • pp.68-74
    • /
    • 2008
  • Application areas of Knowledge Discovery in Database(KDD) have been expanded to many R&D management processes including technology trends analysis, forecasting and evaluation etc. Established research field such as informetrics (or scientometrics) has utilized techniques or methods of KDD. Various systems have been developed to support works of analyzing large-scale R&D related databases such as patent DB or bibliographic DB by a few researchers or institutions. But extant systems have some problems for korean users to use. Their prices is not moderate, korean language processing is impossible, and user's demands not reflected. To solve these problems, Korea Institute of Science and Technology Information(KISTI) developed stand-alone type information analysis system named as KnowledgeMatrix. KnowledgeMatrix system offer various functions to analyze retrieved data set from databases. KnowledgeMatrix's main operation unit is composed of user-defined lists and matrix generation, cluster analysis, visualization, data pre-processing. Matrix generation unit help extract information items which will be analyzed, and calculate occurrence, co-occurrence, proximity of the items. Cluster analysis unit enable matrix data to be clustered by hierarchical or non-hierarchical clustering methods and present tree-type structure of clustered data. Visualization unit offer various methods such as chart, FDP, strategic diagram and PFNet. Data pre-processing unit consists of data import editor, string editor, thesaurus editor, grouping method, field-refining methods and sub-dataset generation methods. KnowledgeMatrix show better performances and offer more various functions than extant systems.

Calculating coniferous tree coverage using unmanned aerial vehicle photogrammetry

  • Ivosevic, Bojana;Han, Yong-Gu;Kwon, Ohseok
    • Journal of Ecology and Environment
    • /
    • v.41 no.3
    • /
    • pp.85-92
    • /
    • 2017
  • Unmanned aerial vehicles (UAVs) are a new and yet constantly developing part of forest inventory studies and vegetation-monitoring fields. Covering large areas, their extensive usage has saved time and money for researchers and conservationists to survey vegetation for various data analyses. Post-processing imaging software has improved the effectiveness of UAVs further by providing 3D models for accurate visualization of the data. We focus on determining the coniferous tree coverage to show the current advantages and disadvantages of the orthorectified 2D and 3D models obtained from the image photogrammetry software, Pix4Dmapper Pro-Non-Commercial. We also examine the methodology used for mapping the study site, additionally investigating the spread of coniferous trees. The collected images were transformed into 2D black and white binary pixel images to calculate the coverage area of coniferous trees in the study site using MATLAB. The research was able to conclude that the 3D model was effective in perceiving the tree composition in the designated site, while the orthorectified 2D map is appropriate for the clear differentiation of coniferous and deciduous trees. In its conclusion, the paper will also be able to show how UAVs could be improved for future usability.

Web-based Application Service Management System for Fault Monitoring

  • Min, Sang-Cheol;Chung, Tai-Myoung;Park, Hyoung-Woo;Lee, Kyung-Ha;Pang, Kee-Hong
    • Journal of Electrical Engineering and information Science
    • /
    • v.2 no.6
    • /
    • pp.64-73
    • /
    • 1997
  • Network technology has been developed for very high-speed networking and multimedia data whose characteristics are the continuous and bursty transmission as well as a large amount of data. With this trend users wish to view the information about the application services as well as network devices and system hardware. However, it is rarely available for the users the information of performance or faults of the application services. Most of information is limited to the information related network devices or system hardware. Furthermore, users expect the best services without knowing the service environments in the network and there is no good way of delivering the service related problems and fault information of application services in a high speed network yet. In this paper we present a web-based application management system that we have developed for the past year. It includes a method to build an agent system that uses an existing network management standards, SNMP MIB and SNMP protocols. The user interface of the system is also developed to support visualization effects with web-based Java interface which offers a convenient way not only to access management information but also to control networked applications.

  • PDF

Advanced signal processing for enhanced damage detection with piezoelectric wafer active sensors

  • Yu, Lingyu;Giurgiutiu, Victor
    • Smart Structures and Systems
    • /
    • v.1 no.2
    • /
    • pp.185-215
    • /
    • 2005
  • Advanced signal processing techniques have been long introduced and widely used in structural health monitoring (SHM) and nondestructive evaluation (NDE). In our research, we applied several signal processing approaches for our embedded ultrasonic structural radar (EUSR) system to obtain improved damage detection results. The EUSR algorithm was developed to detect defects within a large area of a thin-plate specimen using a piezoelectric wafer active sensor (PWAS) array. In the EUSR, the discrete wavelet transform (DWT) was first applied for signal de-noising. Secondly, after constructing the EUSR data, the short-time Fourier transform (STFT) and continuous wavelet transform (CWT) were used for the time-frequency analysis. Then the results were compared thereafter. We eventually chose continuous wavelet transform to filter out from the original signal the component with the excitation signal's frequency. Third, cross correlation method and Hilbert transform were applied to A-scan signals to extract the time of flight (TOF) of the wave packets from the crack. Finally, the Hilbert transform was again applied to the EUSR data to extract the envelopes for final inspection result visualization. The EUSR system was implemented in LabVIEW. Several laboratory experiments have been conducted and have verified that, with the advanced signal processing approaches, the EUSR has enhanced damage detection ability.

A Study on the Statistical Status of By-products from Korean Seafood processing for Utilization of Biomaterials (바이오소재 활용을 위한 국내 수산가공부산물의 통계 현황 연구)

  • Soeon, Ahn;Duckhee, Jang;Do-Hyung, Kang
    • Journal of Marine Bioscience and Biotechnology
    • /
    • v.14 no.2
    • /
    • pp.124-132
    • /
    • 2022
  • By-products from fisheries produced in Korea are of the same industrial material as imported raw materials and are valuable resources for marine bioindustries. Securing raw materials for the mass production of functional materials is one of the main objectives for marine bioindustrial development. The use of fishery by-products as raw materials is anticipated to increase rapidly as the biomarket is growing into a promising industry. In this study, data were acquired from an open-source environment to perform exploratory data analysis, and various visualization methods were used to compare fishery production to the production of marine processed products in the year 2020. This study suggested that the amount of seafood processing, types of processing items, and areas where fishery processing residue is generated, should be able to secure hygienic raw material supply in large quantities. Thus far, it has been found that the Gyeonggi-do and Busan province, where HACCP-certified processing facilities are concentrated, and the local government Seafood Cluster and the Smart Aquaculture Cluster are at the forefront of stable, mass production of raw materials.