• Title/Summary/Keyword: Big Data Computing

Search Result 478, Processing Time 0.028 seconds

Adaptive Resource Management Method base on ART in Cloud Computing Environment (클라우드 컴퓨팅 환경에서 빅데이터 처리를 위한 ART 기반의 적응형 자원관리 방법)

  • Cho, Kyucheol;Kim, JaeKwon
    • Journal of the Korea Society for Simulation
    • /
    • v.23 no.4
    • /
    • pp.111-119
    • /
    • 2014
  • The cloud environment need resource management method that to enable the big data issue and data analysis technology. Existing resource management uses the limited calculation method, therefore concentrated the resource bias problem. To solve this problem, the resource management requires the learning-based scheduling using resource history information. In this paper, we proposes the ART (Adaptive Resonance Theory)-based adaptive resource management. Our proposed method assigns the job to the suitable method with the resource monitoring and history management in cloud computing environment. The proposed method utilizes the unsupervised learning method. Our goal is to improve the data processing and service stability with the adaptive resource management. The propose method allow the systematic management, and utilize the available resource efficiently.

Where and Why? A Novel Approach for Prioritizing Implementation Points of Public CCTVs using Urban Big Data

  • Ji Hye Park;Daehwan Kim;Keon Chul Park
    • Journal of Internet Computing and Services
    • /
    • v.24 no.5
    • /
    • pp.97-106
    • /
    • 2023
  • Citizens' demand for public CCTVs continues to rise, along with an increase in variouscrimes and social problems in cities. In line with the needs of citizens, the Seoul Metropolitan Government began installing CCTV cameras in 2010, and the number of new installations has increased by over 10% each year. As the large surveillance system represents a substantial budget item for the city, decision-making on location selection should be guided by reasonable standards. The purpose of this study is to improve the existing related models(such as public CCTV priority location analysis manuals) to establish the methodology foranalyzing priority regions ofSeoul-type public CCTVs and propose new mid- to long-term installation goals. Additionally, using the improved methodology, we determine the CCTV priority status of 25 autonomous districts across Seoul and calculate the goals. Through its results, this study suggests improvements to existing models by addressing their limitations, such as the sustainability of input data, the conversion of existing general-purpose models to urban models, and the expansion of basic local government-level models to metropolitan government levels. The results can also be applied to other metropolitan areas and are used by the Seoul Metropolitan Government in its CCTV operation policy

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.9
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

Range Segmentation of Dynamic Offloading (RSDO) Algorithm by Correlation for Edge Computing

  • Kang, Jieun;Kim, Svetlana;Kim, Jae-Ho;Sung, Nak-Myoung;Yoon, Yong-Ik
    • Journal of Information Processing Systems
    • /
    • v.17 no.5
    • /
    • pp.905-917
    • /
    • 2021
  • In recent years, edge computing technology consists of several Internet of Things (IoT) devices with embedded sensors that have improved significantly for monitoring, detection, and management in an environment where big data is commercialized. The main focus of edge computing is data optimization or task offloading due to data and task-intensive application development. However, existing offloading approaches do not consider correlations and associations between data and tasks involving edge computing. The extent of collaborative offloading segmented without considering the interaction between data and task can lead to data loss and delays when moving from edge to edge. This article proposes a range segmentation of dynamic offloading (RSDO) algorithm that isolates the offload range and collaborative edge node around the edge node function to address the offloading issue.The RSDO algorithm groups highly correlated data and tasks according to the cause of the overload and dynamically distributes offloading ranges according to the state of cooperating nodes. The segmentation improves the overall performance of edge nodes, balances edge computing, and solves data loss and average latency.

Designing a Crime-Prevention System by Converging Big Data and IoT

  • Jeon, Jin-ho;Jeong, Seung-Ryul
    • Journal of Internet Computing and Services
    • /
    • v.17 no.3
    • /
    • pp.115-128
    • /
    • 2016
  • Recently, converging Big Data and IoT(Internet of Things)has become mainstream, and public sector is no exception. In particular, this combinationis applicable to crime prevention in Korea. Crime prevention has evolved from CPTED (Crime Prevention through Environmental Design) to ubiquitous crime prevention;however, such a physical engineering method has the limitation, for instance, unexpected exposureby CCTV installed on the street, and doesn't have the function that automatically alarms passengers who pass through a criminal zone.To overcome that, this paper offers a crime prevention method using Big Data from public organizations along with IoT. We expect this work will help construct an intelligent crime-prevention system to protect the weak in our society.

Emerging Internet Technology & Service toward Korean Government 3.0

  • Song, In Kuk
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.8 no.2
    • /
    • pp.540-546
    • /
    • 2014
  • Recently a new government has announced an action plan known as the government 3.0, which aims to provide customized services for individual people, generate more jobs and support creative economy. Leading on from previous similar initiatives, the new scheme seeks to focus on open, share, communicate, and collaborate. In promoting Government 3.0, the crucial factor might be how to align the core services and policies of Government 3.0 with correspoding technologies. The paper describes the concepts and features of Government 3.0, identifies emerging Internet-based technologies and services toward the initiative, and finally provides improvement plans for Government 3.0. As a result, 10 issues to be brought together include: Smart Phone Applications and Service, Mobile Internet Computing and Application, Wireless and Sensor Network, Security & Privacy in Internet, Energy-efficient Computing & Smart Grid, Multimedia & Image Processing, Data Mining and Big Data, Software Engineering, Internet Business related Policy, and Management of Internet Application.

Design of Distributed Cloud System for Managing large-scale Genomic Data

  • Seine Jang;Seok-Jae Moon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.16 no.2
    • /
    • pp.119-126
    • /
    • 2024
  • The volume of genomic data is constantly increasing in various modern industries and research fields. This growth presents new challenges and opportunities in terms of the quantity and diversity of genetic data. In this paper, we propose a distributed cloud system for integrating and managing large-scale gene databases. By introducing a distributed data storage and processing system based on the Hadoop Distributed File System (HDFS), various formats and sizes of genomic data can be efficiently integrated. Furthermore, by leveraging Spark on YARN, efficient management of distributed cloud computing tasks and optimal resource allocation are achieved. This establishes a foundation for the rapid processing and analysis of large-scale genomic data. Additionally, by utilizing BigQuery ML, machine learning models are developed to support genetic search and prediction, enabling researchers to more effectively utilize data. It is expected that this will contribute to driving innovative advancements in genetic research and applications.

Leveraging Big Data for Spark Deep Learning to Predict Rating

  • Mishra, Monika;Kang, Mingoo;Woo, Jongwook
    • Journal of Internet Computing and Services
    • /
    • v.21 no.6
    • /
    • pp.33-39
    • /
    • 2020
  • The paper is to build recommendation systems leveraging Deep Learning and Big Data platform, Spark to predict item ratings of the Amazon e-commerce site. Recommendation system in e-commerce has become extremely popular in recent years and it is very important for both customers and sellers in daily life. It means providing the users with products and services they are interested in. Therecommendation systems need users' previous shopping activities and digital footprints to make best recommendation purpose for next item shopping. We developed the recommendation models in Amazon AWS Cloud services to predict the users' ratings for the items with the massive data set of Amazon customer reviews. We also present Big Data architecture to afford the large scale data set for storing and computation. And, we adopted deep learning for machine learning community as it is known that it has higher accuracy for the massive data set. In the end, a comparative conclusion in terms of the accuracy as well as the performance is illustrated with the Deep Learning architecture with Spark ML and the traditional Big Data architecture, Spark ML alone.

An Empirical Study of Implementation and Application of Mold Life Cycle Management Information System In the Cloud Computing Environment (클라우드 컴퓨팅 환경에서 금형 수명주기관리 정보시스템 구축 및 적용의 실증적 연구)

  • Koh, Joon-Cheol;Nam, Seung-Done;Kim, Kyung-Sik
    • Journal of the Korea Safety Management & Science
    • /
    • v.16 no.4
    • /
    • pp.331-341
    • /
    • 2014
  • Internet of Thing(IoT), which is recently talked about with the development of information and communication technology, provides big data to all nodes such as companies and homes, means of transportation etc. by connecting all things with all people through the integrated global network and connecting all actual aspects of economic and social life with Internet of Thing through sensor and software. Defining Internet of Thing, it plays the role of a connector of providing various information required for the decision-making of companies in the cloud computing environment for the Insight usage by collecting and storing Raw Data of the production site through the sensor network and extracting big data in which data is accumulated and Insight through this. In addition, as the industry showing the largest linkage with other root industries among root industries, the mold industry is the core technology for controlling the quality and performance of the final product and realizing the commercialization of new industry such as new growth power industry etc. Recently, awareness on the mold industry is changing from the structure of being labor-intensive, relying on the experience of production workers and repeating modification without the concept of cost to technology-intensive, digitization, high intellectualization due to technology combination according to IT convergence. This study, therefore, is to provide a golden opportunity to increase the direct and indirect expected effects in poor management activities of small businesses by actually implementing and managing the entire process of mold life cycle to information system from mold planning to mass production and preservation by building SME(small and medium-sized enterprises)-type mold life cycle management information system in the cloud computing environment and applying it to the production site.

Evolution of ICT Ecosystem and Mobile Telcos' Counterstrategies (ICT 생태계 변화에 따른 국내 이동통신 사업자의 대응 전략에 대한 연구)

  • Kim, Dong Ju;Kang, Mincheol
    • Journal of Information Technology and Architecture
    • /
    • v.10 no.2
    • /
    • pp.197-209
    • /
    • 2013
  • This study analyzes the nature of consumers and smart phones as well as its limitations that domestic mobile communication companies confront. According to the analysis results, emerging technologies such as 5G communication, pervasive computing, augmented reality, and big data seem to have significant effect on the ICT ecosystem in the near future. Based on the results, this study suggests four counterstrategies for domestic mobile communication companies: big data strategy, preparation of things acting as a main communication agent, new service platform development, and 'total life care service provider' strategy.