• Title/Summary/Keyword: Distributed Server

Search Result 849, Processing Time 0.03 seconds

Job-related analysis and visualization using big data distributed processing system (빅데이터를 활용한 직업관련 분석 및 시각화)

  • Choi, Dong-Cheol;Choi, Nakjin;Kim, Min-Seok;Park, Jun-wook;Lee, Jun-Dong
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2020.07a
    • /
    • pp.249-251
    • /
    • 2020
  • 본 논문에서는 코로나바이러스감염증19 사태가 국내 취업시장에 어떠한 영향을 미쳤는지에 대해 알아보기 위하여 빅데이터를 활용한 직업 관련 분석 및 시각화를 수행하였다. 빅데이터를 위한 기본 자료는 통계청 자료와 워크넷 Open API를 활용하였으며, 빅데이터 처리 과정을 거쳐 결과값을 예측을 시도하였다. 2020년도 워크넷 Open API를 통해 고용수와 통계청 자료를 통해 비교 분석 및 시각화를 실시하였고, 08년~20년 취업자수를 통해 시계열 분석 및 예측을 진행해 앞으로의 횡보를 예상해보았다. 분석한 결과 19년, 20년도를 비교 분석했을 때에는 크게 차이가 나지 않았다. 추가적으로 시계열 분석기법을 활용해 보았을 때 매년 고용수는 전체적으로 증가하고 4월에는 감소, 7월에는 증가하는 추세가 나왔다. 코로나바이러스감염증19 사태로 인해 공공기관과 언택트 시대에 따른 화상회의나 재택근무로 인해 운수·통신 취업률은 상승한다는 결과값이 도출되었고, 자영업이나 서비스 직업 등은 다른 직종에 비해 큰 감소를 보여줬으나 국가 경제 활성화에 따른 고용수는 점차 증가할 것이라 예측된다.

  • PDF

A Technique for Protecting Android Applications using Executable Code Encryption and Integrity Verification (실행코드 암호화 및 무결성 검증을 적용한 안드로이드앱 보호 기법)

  • Shim, HyungJoon;Cho, Sangwook;Jeong, Younsik;Lee, Chanhee;Han, Sangchul;Cho, Seong-je
    • Journal of Software Assessment and Valuation
    • /
    • v.10 no.1
    • /
    • pp.19-26
    • /
    • 2014
  • In this paper, we propose a method for protecting Android applications against reverse engineering attacks. In this method, the server encrypts the original executable code (DEX) included in an APK file, inserts into the APK file a stub code that decrypts the encrypted DEX later at run-time, and distributes the modified APK file. The stub code includes an integrity validation code to detect attacks on itself. When a user installs and executes the APK file, the stub code verifies the integrity of itself, decrypts the encrypted DEX, and loads it dynamically to execute. Since the original DEX is distributed as an encrypted one, we can effectively protect the intellectual property. Further, by verifying the integrity of the stub code, we can prevent malicious users from bypassing our method. We applied the method to 15 Android apps, and evaluated its effectiveness. We confirmed that 13 out of them operates normally.

A Study on the Efficiency Method of Location-Based Control Function Applied to MDM Applications (MDM 앱에 적용되는 위치 기반 통제 기능의 효율화 방법에 관한 연구)

  • ChangIk Oh;SuGil Hwang;Hyun Son;Dongho Kim
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.23 no.6
    • /
    • pp.171-177
    • /
    • 2023
  • When configuring MDM services only with applications without a central control server, a method that allows security control to be released based on GPS signals in areas sufficiently far from the security area of the workplace is generally applied by embedding and distributing location information of workplaces in MDM applications. This study proposed a method in which each individual directly enters the location information of the security control area needed for them in the distributed app and maintains it as setting information without embedding location information of a specific area such as work in the MDM applications. The method proposed in this study can improve universality, compatibility and the security control level of MDM services, and minimize deployment costs.

Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku
    • Journal of Internet Computing and Services
    • /
    • v.14 no.6
    • /
    • pp.71-84
    • /
    • 2013
  • Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

A Study on Integration of Healthcare Information Systems based on P2P in Distributed Environment (분산환경에서의 P2P기반 보건의료분야 정보시스템 통합에 관한 연구)

  • Park, Yong-Min;Oh, Young-Hwan
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.48 no.2
    • /
    • pp.36-42
    • /
    • 2011
  • The current healthcare information systems field to meet the growing demand for healthcare for a variety of building systems and operation, and subsequent information on the budget continues to increase, but the current system, although the association link between the various systems made does not, with organizations with information about each of the standardization and real-time network status data do not consist of various materials, such as insufficient to provide real-time issues have been raised. This paper proposes a Integrated information system on Healthcare based on JXTA to solve problems mentioned above. Until now, in a network environment for data storage and management is the most widely used server-intensive structure, while an increase in users and traffic difficulties in data management and communications services to handle the growing number of servers increase faster than information associated with the cost savings, P2P model in terms of efficient data management has emerged as a new solution. Therefore this paper designs a platform for Integrated information system on Healthcare based on JXTA as a method to integrate health information data and services, and then proves that the new information system on healthcare based on JXTA is the suitable model.

S-FDS : a Smart Fire Detection System based on the Integration of Fuzzy Logic and Deep Learning (S-FDS : 퍼지로직과 딥러닝 통합 기반의 스마트 화재감지 시스템)

  • Jang, Jun-Yeong;Lee, Kang-Woon;Kim, Young-Jin;Kim, Won-Tae
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.54 no.4
    • /
    • pp.50-58
    • /
    • 2017
  • Recently, some methods of converging heterogeneous fire sensor data have been proposed for effective fire detection, but the rule-based methods have low adaptability and accuracy, and the fuzzy inference methods suffer from detection speed and accuracy by lack of consideration for images. In addition, a few image-based deep learning methods were researched, but it was too difficult to rapidly recognize the fire event in absence of cameras or out of scope of a camera in practical situations. In this paper, we propose a novel fire detection system combining a deep learning algorithm based on CNN and fuzzy inference engine based on heterogeneous fire sensor data including temperature, humidity, gas, and smoke density. we show it is possible for the proposed system to rapidly detect fire by utilizing images and to decide fire in a reliable way by utilizing multi-sensor data. Also, we apply distributed computing architecture to fire detection algorithm in order to avoid concentration of computing power on a server and to enhance scalability as a result. Finally, we prove the performance of the system through two experiments by means of NIST's fire dynamics simulator in both cases of an explosively spreading fire and a gradually growing fire.

Reconfiguration of Apache Storm for InfiniBand Communications (InfiniBand RDMA 통신을 위한 Apache Storm의 재구성)

  • Yang, Seokwoo;Son, Siwoon;Moon, Yang-Sae
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.7 no.8
    • /
    • pp.297-306
    • /
    • 2018
  • In this paper, we address how to apply Apache Storm, a distributed stream processing framework, to InfiniBand, a high performance communication device. An easy way to run Storm on InfiniBand is to simply use IPoIP (IP over InfiniBand). However, this method causes a serious CPU load on the node, which is caused by frequent context switches and buffer copies. To solve this problem, we propose a new communication method using InfiniBand's Remote Direct Memory Access (RDMA) function in Storm. First, we design and implement RJ-Netty (RDMA/JXIO Netty), a new framework that replaces Netty, the legacy framework, to exploit RDMA functionality. Second, we reimplement the related classes so that Storm can use both existing Netty and new RJ-Netty. Third, we extend the JXIO server functionality so as to support multi-threading to maximize the performance of RJ-Netty. Experimental results show that the proposed RJ-Netty significantly reduces CPU load while improving message throughput compared to IPoIB as well as Ethernet. This paper is the first attempt to run Apache Storm on InfiniBand, and we believe that it is an excellent research result that improves the performance of Storm by using InfiniBand RDMA.

Design and Implementation of a Metadata Structure for Large-Scale Shared-Disk File System (대용량 공유디스크 파일 시스템에 적합한 메타 데이타 구조의 설계 및 구현)

  • 이용주;김경배;신범주
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.30 no.1
    • /
    • pp.33-49
    • /
    • 2003
  • Recently, there have been large storage demands for manipulating multimedia data. To solve the tremendous storage demands, one of the major researches is the SAN(Storage Area Network) that provides the local file requests directly from shared-disk storage and also eliminates the server bottlenecks to performance and availability. SAN also improve the network latency and bandwidth through new channel interface like FC(Fibre Channel). But to manipulate the efficient storage network like SAN, traditional local file system and distributed file system are not adaptable and also are lack of researches in terms of a metadata structure for large-scale inode object such as file and directory. In this paper, we describe the architecture and design issues of our shared-disk file system and provide the efficient bitmap for providing the well-formed block allocation in each host, extent-based semi flat structure for storing large-scale file data, and two-phase directory structure of using Extendible Hashing. Also we describe a detailed algorithm for implementing the file system's device driver in Linux Kernel and compare our file system with the general file system like EXT2 and shard disk file system like GFS in terms of file creation, directory creation and I/O rate.

A Study on Lightweight Block Cryptographic Algorithm Applicable to IoT Environment (IoT 환경에 적용 가능한 경량화 블록 암호알고리즘에 관한 연구)

  • Lee, Seon-Keun
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.3
    • /
    • pp.1-7
    • /
    • 2018
  • The IoT environment provides an infinite variety of services using many different devices and networks. The development of the IoT environment is directly proportional to the level of security that can be provided. In some ways, lightweight cryptography is suitable for IoT environments, because it provides security, higher throughput, low power consumption and compactness. However, it has the limitation that it must form a new cryptosystem and be used within a limited resource range. Therefore, it is not the best solution for the IoT environment that requires diversification. Therefore, in order to overcome these disadvantages, this paper proposes a method suitable for the IoT environment, while using the existing block cipher algorithm, viz. the lightweight cipher algorithm, and keeping the existing system (viz. the sensing part and the server) almost unchanged. The proposed BCL architecture can perform encryption for various sensor devices in existing wire/wireless USNs (using) lightweight encryption. The proposed BCL architecture includes a pre/post-processing part in the existing block cipher algorithm, which allows various scattered devices to operate in a daisy chain network environment. This characteristic is optimal for the information security of distributed sensor systems and does not affect the neighboring network environment, even if hacking and cracking occur. Therefore, the BCL architecture proposed in the IoT environment can provide an optimal solution for the diversified IoT environment, because the existing block cryptographic algorithm, viz. the lightweight cryptographic algorithm, can be used.

Intelligent Web Crawler for Supporting Big Data Analysis Services (빅데이터 분석 서비스 지원을 위한 지능형 웹 크롤러)

  • Seo, Dongmin;Jung, Hanmin
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.12
    • /
    • pp.575-584
    • /
    • 2013
  • Data types used for big-data analysis are very widely, such as news, blog, SNS, papers, patents, sensed data, and etc. Particularly, the utilization of web documents offering reliable data in real time is increasing gradually. And web crawlers that collect web documents automatically have grown in importance because big-data is being used in many different fields and web data are growing exponentially every year. However, existing web crawlers can't collect whole web documents in a web site because existing web crawlers collect web documents with only URLs included in web documents collected in some web sites. Also, existing web crawlers can collect web documents collected by other web crawlers already because information about web documents collected in each web crawler isn't efficiently managed between web crawlers. Therefore, this paper proposed a distributed web crawler. To resolve the problems of existing web crawler, the proposed web crawler collects web documents by RSS of each web site and Google search API. And the web crawler provides fast crawling performance by a client-server model based on RMI and NIO that minimize network traffic. Furthermore, the web crawler extracts core content from a web document by a keyword similarity comparison on tags included in a web documents. Finally, to verify the superiority of our web crawler, we compare our web crawler with existing web crawlers in various experiments.