• Title/Summary/Keyword: Big Data Processing Technology

Search Result 388, Processing Time 0.031 seconds

Study of Improved CNN Algorithm for Object Classification Machine Learning of Simple High Resolution Image (고해상도 단순 이미지의 객체 분류 학습모델 구현을 위한 개선된 CNN 알고리즘 연구)

  • Hyeopgeon Lee;Young-Woon Kim
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.16 no.1
    • /
    • pp.41-49
    • /
    • 2023
  • A convolutional neural network (CNN) is a representative algorithm for implementing artificial neural networks. CNNs have improved on the issues of rapid increase in calculation amount and low object classification rates, which are associated with a conventional multi-layered fully-connected neural network (FNN). However, because of the rapid development of IT devices, the maximum resolution of images captured by current smartphone and tablet cameras has reached 108 million pixels (MP). Specifically, a traditional CNN algorithm requires a significant cost and time to learn and process simple, high-resolution images. Therefore, this study proposes an improved CNN algorithm for implementing an object classification learning model for simple, high-resolution images. The proposed method alters the adjacency matrix value of the pooling layer's max pooling operation for the CNN algorithm to reduce the high-resolution image learning model's creation time. This study implemented a learning model capable of processing 4, 8, and 12 MP high-resolution images for each altered matrix value. The performance evaluation result showed that the creation time of the learning model implemented with the proposed algorithm decreased by 36.26% for 12 MP images. Compared to the conventional model, the proposed learning model's object recognition accuracy and loss rate were less than 1%, which is within the acceptable error range. Practical verification is necessary through future studies by implementing a learning model with more varied image types and a larger amount of image data than those used in this study.

Development of smart car intelligent wheel hub bearing embedded system using predictive diagnosis algorithm

  • Sam-Taek Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.10
    • /
    • pp.1-8
    • /
    • 2023
  • If there is a defect in the wheel bearing, which is a major part of the car, it can cause problems such as traffic accidents. In order to solve this problem, big data is collected and monitoring is conducted to provide early information on the presence or absence of wheel bearing failure and type of failure through predictive diagnosis and management technology. System development is needed. In this paper, to implement such an intelligent wheel hub bearing maintenance system, we develop an embedded system equipped with sensors for monitoring reliability and soundness and algorithms for predictive diagnosis. The algorithm used acquires vibration signals from acceleration sensors installed in wheel bearings and can predict and diagnose failures through big data technology through signal processing techniques, fault frequency analysis, and health characteristic parameter definition. The implemented algorithm applies a stable signal extraction algorithm that can minimize vibration frequency components and maximize vibration components occurring in wheel bearings. In noise removal using a filter, an artificial intelligence-based soundness extraction algorithm is applied, and FFT is applied. The fault frequency was analyzed and the fault was diagnosed by extracting fault characteristic factors. The performance target of this system was over 12,800 ODR, and the target was met through test results.

Subgraph Searching Scheme Based on Path Queries in Distributed Environments (분산 환경에서 경로 질의 기반 서브 그래프 탐색 기법)

  • Kim, Minyoung;Choi, Dojin;Park, Jaeyeol;Kim, Yeondong;Lim, Jongtae;Bok, Kyoungsoo;Choi, Han Suk;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.1
    • /
    • pp.141-151
    • /
    • 2019
  • A network of graph data structure is used in many applications to represent interactions between entities. Recently, as the size of the network to be processed due to the development of the big data technology is getting larger, it becomes more difficult to handle it in one server, and thus the necessity of distributed processing is also increasing. In this paper, we propose a distributed processing system for efficiently performing subgraph and stores. To reduce unnecessary searches, we use statistical information of the data to determine the search order through probabilistic scoring. Since the relationship between the vertex and the degree of the graph network may show different characteristics depending on the type of data, the search order is determined by calculating a score to reduce unnecessary search through a different scoring method for a graph having various distribution characteristics. The graph is sequentially searched in the distributed servers according to the determined order. In order to demonstrate the superiority of the proposed method, performance comparison with the existing method was performed. As a result, the search time is improved by about 3 ~ 10% compared with the existing method.

A Study on Adaptive Learning Model for Performance Improvement of Stream Analytics (실시간 데이터 분석의 성능개선을 위한 적응형 학습 모델 연구)

  • Ku, Jin-Hee
    • Journal of Convergence for Information Technology
    • /
    • v.8 no.1
    • /
    • pp.201-206
    • /
    • 2018
  • Recently, as technologies for realizing artificial intelligence have become more common, machine learning is widely used. Machine learning provides insight into collecting large amounts of data, batch processing, and taking final action, but the effects of the work are not immediately integrated into the learning process. In this paper proposed an adaptive learning model to improve the performance of real-time stream analysis as a big business issue. Adaptive learning generates the ensemble by adapting to the complexity of the data set, and the algorithm uses the data needed to determine the optimal data point to sample. In an experiment for six standard data sets, the adaptive learning model outperformed the simple machine learning model for classification at the learning time and accuracy. In particular, the support vector machine showed excellent performance at the end of all ensembles. Adaptive learning is expected to be applicable to a wide range of problems that need to be adaptively updated in the inference of changes in various parameters over time.

Compression Conversion and Storing of Large RDF datasets based on MapReduce (맵리듀스 기반 대량 RDF 데이터셋 압축 변환 및 저장 방법)

  • Kim, InA;Lee, Kyong-Ha;Lee, Kyu-Chul
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.4
    • /
    • pp.487-494
    • /
    • 2022
  • With the recent demand for analysis using data, the size of the knowledge graph, which is the data to be analyzed, gradually increased, reaching about 82 billion edges when extracted from the web as a knowledge graph. A lot of knowledge graphs are represented in the form of Resource Description Framework (RDF), which is a standard of W3C for representing metadata for web resources. Because of the characteristics of RDF, existing RDF storages have the limitations of processing time overhead when converting and storing large amounts of RDF data. To resolve these limitations, in this paper, we propose a method of compressing and converting large amounts of RDF data into integer IDs using MapReduce, and vertically partitioning and storing them. Our proposed method demonstrated a high performance improvement of up to 25.2 times compared to RDF-3X and up to 3.7 times compared to H2RDF+.

Energy-Efficient Subpaging for the MRAM-based SSD File System (MRAM 기반 SSD 파일 시스템의 에너지 효율적 서브페이징)

  • Lee, JaeYoul;Han, Jae-Il;Kim, Young-Man
    • Journal of Information Technology Services
    • /
    • v.12 no.4
    • /
    • pp.369-380
    • /
    • 2013
  • The advent of the state-of-the-art technologies such as cloud computing and big data processing stimulates the provision of various new IT services, which implies that more servers are required to support them. However, the need for more servers will lead to more energy consumption and the efficient use of energy in the computing environment will become more important. The next generation nonvolatile RAM has many desirable features such as byte addressability, low access latency, high density and low energy consumption. There are many approaches to adopt them especially in the area of the file system involving storage devices, but their focus lies on the improvement of system performance, not on energy reduction. This paper suggests a novel approach for energy reduction in which the MRAM-based SSD is utilized as a storage device instead of the hard disk and a downsized page is adopted instead of the 4KB page that is the size of a page in the ordinary file system. The simulation results show that energy efficiency of a new approach is very effective in case of accessing the small number of bytes and is improved up to 128 times better than that of NAND Flash memory.

Managerial Factors Influencing Dose Reduction of the Nozzle Dam Installation and Removal Tasks Inside a Steam Generator Water Chamber (증기발생기 수실 노즐댐 설치 및 제거작업의 피폭선량 저감에 영향을 주는 관리요인에 관한 연구)

  • Lee, Dhong Ha
    • Journal of the Ergonomics Society of Korea
    • /
    • v.36 no.5
    • /
    • pp.559-568
    • /
    • 2017
  • Objective: The aim of this study is to investigate the effective managerial factors influencing dose reduction of the nozzle dam installation and removal tasks ranking within top 3 in viewpoint of average collective dose of nuclear power plant maintenance job. Background: International Commission on Radiation Protection (ICRP) recommended to reduce unnecessary dose and to minimize the necessary dose on the participants of maintenance job in radiation fields. Method: Seven sessions of nozzle dam installation and removal task logs yielded a multiple regression model with collective dose as a dependent variable and work time, number of participants, space doses before and after shield as independent variables. From the sessions in which a significant reduction in collective dose occurred, the effective managerial factors were elicited. Results: Work time was the most important factor contributing to collective dose reduction of nozzle dam installation and removal task. Introduction of new technology in nozzle dam design or maintenance job is the most important factor for work time reduction. Conclusion: With extended task logs and big data processing technique, the more accurate prediction model illustrating the relationship between collective dose reduction and effective managerial factors would be developed. Application: The effective managerial factors will be useful to reduce collective dose of decommissioning tasks as well as regular preventive maintenance tasks for a nuclear power plant.

Mini-review on VO2-based Sensors Utilizing Metal-insulator Transition

  • Hyeongyu Gim;Minho Lee;Woojin Hong;Kootak Hong
    • Journal of Sensor Science and Technology
    • /
    • v.33 no.5
    • /
    • pp.265-273
    • /
    • 2024
  • With the advent of artificial intelligence and Internet of Things, demands for high-performance sensors with high sensitivity and ultrafast response for big data acquisition and processing have increased. VO2, a strongly correlated material, has been shown to exhibit a reversible and abrupt resistance change in the sub-nanosecond scale through a phase transition from an insulating to a metallic state at 68℃. The metal-insulator transition (MIT) of VO2 provides the potential for the development of highly sensitive and ultrafast high-performance sensors. This is because it can be triggered by various external stimuli, such as heat, light, gas adsorption/desorption, and strain. Therefore, attempts have been made to develop high-performance sensors by controlling the MIT of VO2 in response to external stimuli. This study reviewed recent progress in various VO2-based sensors that utilize MIT, including photodetectors, gas sensors, and strain sensors. This review is expected to serve as an overview of the approaches for controlling the MIT behavior of VO2 and provide insights into the design of high-performance sensors that exploit MIT.

De-cloaking Malicious Activities in Smartphones Using HTTP Flow Mining

  • Su, Xin;Liu, Xuchong;Lin, Jiuchuang;He, Shiming;Fu, Zhangjie;Li, Wenjia
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.6
    • /
    • pp.3230-3253
    • /
    • 2017
  • Android malware steals users' private information, and embedded unsafe advertisement (ad) libraries, which execute unsafe code causing damage to users. The majority of such traffic is HTTP and is mixed with other normal traffic, which makes the detection of malware and unsafe ad libraries a challenging problem. To address this problem, this work describes a novel HTTP traffic flow mining approach to detect and categorize Android malware and unsafe ad library. This work designed AndroCollector, which can automatically execute the Android application (app) and collect the network traffic traces. From these traces, this work extracts HTTP traffic features along three important dimensions: quantitative, timing, and semantic and use these features for characterizing malware and unsafe ad libraries. Based on these HTTP traffic features, this work describes a supervised classification scheme for detecting malware and unsafe ad libraries. In addition, to help network operators, this work describes a fine-grained categorization method by generating fingerprints from HTTP request methods for each malware family and unsafe ad libraries. This work evaluated the scheme using HTTP traffic traces collected from 10778 Android apps. The experimental results show that the scheme can detect malware with 97% accuracy and unsafe ad libraries with 95% accuracy when tested on the popular third-party Android markets.

Current status of food safety detection methods for Smart-HACCP system (스마트-해섭(Smart-HACCP) 적용을 위한 식품안전 검시기술 동향)

  • Lim, Min-Cheol;Woo, Min-Ah;Choi, Sung-Wook
    • Food Science and Industry
    • /
    • v.54 no.4
    • /
    • pp.293-300
    • /
    • 2021
  • Food safety accidents have been increasing by 2% over 5,000 cases every year since 2009. Most people know that the best method to prevent food safety accidents is a quick inspection, but there is a lack of inspection technology that can be used at the non-analytic level to food production and distribution sites. Among the recent on-site diagnostic technologies, the methods for testing gene-based food poisoning bacteria were introduced with the STA technology, which can range from sample to detection. If food safety information can be generated without forgery by directly inspecting food hazard factors by remote, unmanned, not human, pollution sources can be managed by predicting risks more accurately from current big-data and artificial intelligence technology. Since this information processing can be used on smartphones using the current cloud technology, it is judged that it can be used for food safety to small food businesses or catering services.