• Title/Summary/Keyword: Big data Processing

Search Result 1,063, Processing Time 0.025 seconds

A Study of Policy Direction by City and Province through the Prediction of Household Loan Delinquency Rate (가계대출 연체율 예측을 통한 시도별 정책 방향성 연구)

  • Su-jin Lee;Jeong-in Won;Hee-yong Kang;In-seong Lee;Gun Kim;Jin Kim
    • Annual Conference of KIPS
    • /
    • 2023.11a
    • /
    • pp.380-381
    • /
    • 2023
  • 최근 경제침체로 인해 지속되는 연체율 상승의 원인을 지역별 및 시차별로 분석하였다. 독립변수를 가계대출변수, 부동산지수변수, 경제지표변수로 나누었고 통계적 모델링을 통해 총 19 가지 변수로 연체율을 예측하였다. 각 지역마다 상이한 결과가 도출되었는데 이를 바탕으로 지역별 연체율 감소 정책을 제안한다.

A Study on Map Mapping of Individual Vehicle Big Data Based on Space (공간 기반의 개별 차량 대용량 정보 맵핑에 관한 연구)

  • Chong, Kyusoo
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.20 no.5
    • /
    • pp.75-82
    • /
    • 2021
  • The number of traffic accidents is about 230,000, and due to non-recurring congestion and high driving speed, the number of deaths per traffic accident on freeways is more than twice compared to other roads. Currently, traffic information is provided based on nodes and links using the centerline of the road, but it does not provide detailed speed information. Recently, installing sensors for vehicles to monitor obstacles and measure location is becoming common not only for autonomous vehicles but also for ordinary vehicles as well. The analysis using large-capacity location-based data from such sensors enables real time service according to processing speed. This study presents an mapping method for individual vehicle data analysis based on space. The processing speed of large-capacity data was increased by using method which applied a quaternary notation basis partition method that splits into two directions of longitude and latitude respectively. As the space partition was processed, the average speed was similar, but the speed standard deviation gradually decreased, and decrease range became smaller after 9th partition.

Data Source Management using weight table in u-GIS DSMS

  • Kim, Sang-Ki;Baek, Sung-Ha;Lee, Dong-Wook;Chung, Warn-Il;Kim, Gyoung-Bae;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.27-33
    • /
    • 2009
  • The emergences of GeoSensor and researches about GIS have promoted many researches of u-GIS. The disaster application coupled in the u-GIS can apply to monitor accident area and to prevent spread of accident. The application needs the u-GIS DSMS technique to acquire, to process GeoSensor data and to integrate them with GIS data. The u-GIS DSMS must process big and large-volume data stream such as spatial data and multimedia data. Due to the feature of the data stream, in u-GIS DSMS, query processing can be delayed. Moreover, as increasing the input rate of data in the area generating events, the network traffic is increased. To solve this problem, in this paper we describe TRIGGER ACTION clause in CQ on the u-GIS DSMS environment and proposes data source management. Data source weight table controls GES information and incoming data rate. It controls incoming data rate as increasing weight at GES of disaster area. Consequently, it can contribute query processing rate and accuracy

  • PDF

Determination of Fire Risk Assessment Indicators for Building using Big Data (빅데이터를 활용한 건축물 화재위험도 평가 지표 결정)

  • Joo, Hong-Jun;Choi, Yun-Jeong;Ok, Chi-Yeol;An, Jae-Hong
    • Journal of the Korea Institute of Building Construction
    • /
    • v.22 no.3
    • /
    • pp.281-291
    • /
    • 2022
  • This study attempts to use big data to determine the indicators necessary for a fire risk assessment of buildings. Because most of the causes affecting the fire risk of buildings are fixed as indicators considering only the building itself, previously only limited and subjective assessment has been performed. Therefore, if various internal and external indicators can be considered using big data, effective measures can be taken to reduce the fire risk of buildings. To collect the data necessary to determine indicators, a query language was first selected, and professional literature was collected in the form of unstructured data using a web crawling technique. To collect the words in the literature, pre-processing was performed such as user dictionary registration, duplicate literature, and stopwords. Then, through a review of previous research, words were classified into four components, and representative keywords related to risk were selected from each component. Risk-related indicators were collected through analysis of related words of representative keywords. By examining the indicators according to their selection criteria, 20 indicators could be determined. This research methodology indicates the applicability of big data analysis for establishing measures to reduce fire risk in buildings, and the determined risk indicators can be used as reference materials for assessment.

Adaptive Resource Management Method base on ART in Cloud Computing Environment (클라우드 컴퓨팅 환경에서 빅데이터 처리를 위한 ART 기반의 적응형 자원관리 방법)

  • Cho, Kyucheol;Kim, JaeKwon
    • Journal of the Korea Society for Simulation
    • /
    • v.23 no.4
    • /
    • pp.111-119
    • /
    • 2014
  • The cloud environment need resource management method that to enable the big data issue and data analysis technology. Existing resource management uses the limited calculation method, therefore concentrated the resource bias problem. To solve this problem, the resource management requires the learning-based scheduling using resource history information. In this paper, we proposes the ART (Adaptive Resonance Theory)-based adaptive resource management. Our proposed method assigns the job to the suitable method with the resource monitoring and history management in cloud computing environment. The proposed method utilizes the unsupervised learning method. Our goal is to improve the data processing and service stability with the adaptive resource management. The propose method allow the systematic management, and utilize the available resource efficiently.

A Study on the Implementation of Crawling Robot using Q-Learning

  • Hyunki KIM;Kyung-A KIM;Myung-Ae CHUNG;Min-Soo KANG
    • Korean Journal of Artificial Intelligence
    • /
    • v.11 no.4
    • /
    • pp.15-20
    • /
    • 2023
  • Machine learning is comprised of supervised learning, unsupervised learning and reinforcement learning as the type of data and processing mechanism. In this paper, as input and output are unclear and it is difficult to apply the concrete modeling mathematically, reinforcement learning method are applied for crawling robot in this paper. Especially, Q-Learning is the most effective learning technique in model free reinforcement learning. This paper presents a method to implement a crawling robot that is operated by finding the most optimal crawling method through trial and error in a dynamic environment using a Q-learning algorithm. The goal is to perform reinforcement learning to find the optimal two motor angle for the best performance, and finally to maintain the most mature and stable motion about EV3 Crawling robot. In this paper, for the production of the crawling robot, it was produced using Lego Mindstorms with two motors, an ultrasonic sensor, a brick and switches, and EV3 Classroom SW are used for this implementation. By repeating 3 times learning, total 60 data are acquired, and two motor angles vs. crawling distance graph are plotted for the more understanding. Applying the Q-learning reinforcement learning algorithm, it was confirmed that the crawling robot found the optimal motor angle and operated with trained learning, and learn to know the direction for the future research.

Material as a Key Element of Fashion Trend in 2010~2019 - Text Mining Analysis - (패션 트렌트(2010~2019)의 주요 요소로서 소재 - 텍스트마이닝을 통한 분석 -)

  • Jang, Namkyung;Kim, Min-Jeong
    • Fashion & Textile Research Journal
    • /
    • v.22 no.5
    • /
    • pp.551-560
    • /
    • 2020
  • Due to the nature of fashion design that responds quickly and sensitively to changes, accurate forecasting for upcoming fashion trends is an important factor in the performance of fashion product planning. This study analyzed the major phenomena of fashion trends by introducing text mining and a big data analysis method. The research questions were as follows. What is the key term of the 2010SS~2019FW fashion trend? What are the terms that are highly relevant to the key trend term by year? Which terms relevant to the key trend term has shown high frequency in news articles during the same period? Data were collected through the 2010SS~2019FW Pre-Trend data from the leading trend information company in Korea and 45,038 articles searched by "fashion+material" from the News Big Data System. Frequency, correlation coefficient, coefficient of variation and mapping were performed using R-3.5.1. Results showed that the fashion trend information were reflected in the consumer market. The term with the highest frequency in 2010SS~2019FW fashion trend information was material. In trend information, the terms most relevant to material were comfort, compact, look, casual, blend, functional, cotton, processing, metal and functional by year. In the news article, functional, comfort, sports, leather, casual, eco-friendly, classic, padding, culture, and high-quality showed the high frequency. Functional was the only fashion material term derived every year for 10 years. This study helps expand the scope and methods of fashion design research as well as improves the information analysis and forecasting capabilities of the fashion industry.

Efficient Computation of Data Cubes Using MapReduce (맵리듀스를 사용한 데이터 큐브의 효율적인 계산 기법)

  • Lee, Ki Yong;Park, Sojeong;Park, Eunju;Park, Jinkyung;Choi, Yeunjung
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.11
    • /
    • pp.479-486
    • /
    • 2014
  • MapReduce is a programing model used for parallelly processing a large amount of data. To analyze a large amount data, the data cube is widely used, which is an operator that computes group-bys for all possible combinations of given dimension attributes. When the number of dimension attributes is n, the data cube computes $2^n$ group-bys. In this paper, we propose an efficient method for computing data cubes using MapReduce. The proposed method partitions $2^n$ group-bys into $_nC_{{\lceil}n/2{\rceil}}$ batches, and computes those batches in stages using ${\lceil}n/2{\rceil}$ MapReduce jobs. Compared to the existing methods, the proposed method significantly reduces the amount of intermediate data generated by mappers, so that the cost of sorting and transferring those intermediate data is reduced significantly. Consequently, the total processing time for computing a data cube is reduced. Through experiments, we show the efficiency of the proposed method over the existing methods.

A study on Korean language processing using TF-IDF (TF-IDF를 활용한 한글 자연어 처리 연구)

  • Lee, Jong-Hwa;Lee, MoonBong;Kim, Jong-Weon
    • The Journal of Information Systems
    • /
    • v.28 no.3
    • /
    • pp.105-121
    • /
    • 2019
  • Purpose One of the reasons for the expansion of information systems in the enterprise is the increased efficiency of data analysis. In particular, the rapidly increasing data types which are complex and unstructured such as video, voice, images, and conversations in and out of social networks. The purpose of this study is the customer needs analysis from customer voices, ie, text data, in the web environment.. Design/methodology/approach As previous study results, the word frequency of the sentence is extracted as a word that interprets the sentence has better affects than frequency analysis. In this study, we applied the TF-IDF method, which extracts important keywords in real sentences, not the TF method, which is a word extraction technique that expresses sentences with simple frequency only, in Korean language research. We visualized the two techniques by cluster analysis and describe the difference. Findings TF technique and TF-IDF technique are applied for Korean natural language processing, the research showed the value from frequency analysis technique to semantic analysis and it is expected to change the technique by Korean language processing researcher.

Artificial Intelligence Semiconductor and Packaging Technology Trend (인공지능 반도체 및 패키징 기술 동향)

  • Hee Ju Kim;Jae Pil Jung
    • Journal of the Microelectronics and Packaging Society
    • /
    • v.30 no.3
    • /
    • pp.11-19
    • /
    • 2023
  • Recently with the rapid advancement of artificial intelligence (AI) technologies such as Chat GPT, AI semiconductors have become important. AI technologies require the ability to process large volumes of data quickly, as they perform tasks such as big data processing, deep learning, and algorithms. However, AI semiconductors encounter challenges with excessive power consumption and data bottlenecks during the processing of large-scale data. Thus, the latest packaging technologies are required for AI semiconductor computations. In this study, the authors have described packaging technologies applicable to AI semiconductors, including interposers, Through-Silicon-Via (TSV), bumping, Chiplet, and hybrid bonding. These technologies are expected to contribute to enhance the power efficiency and processing speed of AI semiconductors.