• Title/Summary/Keyword: distributed-data processing algorithm

Search Result 181, Processing Time 0.026 seconds

An Extensible Transaction Model for Real-Time Data Processing (실시간 데이타 처리를 위한 확장 가능한 트랜잭션 모델에 관한 연구)

  • 문승진
    • Journal of Internet Computing and Services
    • /
    • v.1 no.2
    • /
    • pp.11-18
    • /
    • 2000
  • In this paper we present a new extensible model based upon the concept of subtransactions in real-time transaction systems. The nested transaction model originally proposed by J. Moss is extended for real-time uniprocessor transaction systems by adding explicit timing constraints. Based upon the model, an integrated concurrency control and scheduling algorithm is developed, that not only guarantees timing constraints of a set of real-time transactions but also maintains consistency of the database. The algorithm is based on the priority ceiling protocol of Sha et al. We prove that the Real-Time Nested Priority Ceiling Protocol prevents unbounded blocking and deadlock, and maintains the serializability of a set of real-time transactions. We use the upper bound on the duration that a transaction can be blocked to show that it is possible to analyze the schedulability of a transaction set using rate-monotonic priority assignment. This work is viewed as a step toward multiprocessor and distributed real-time nested transaction systems. Also, it is possible to be extended to include the real-time multimedia transactions in the emerging web-based database application areas.

  • PDF

Design and Development of Network for Housing Estate Security System

  • Nachin, Awacharin;Mitatha, Somsak;Dejhan, Kobchai;Kirdpipat, Patchanon;Miyanaga, Yoshikazu
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2003.10a
    • /
    • pp.1480-1484
    • /
    • 2003
  • This paper presents the design and development of network for housing estate security system. The system can cover up to 961 houses which can be up to 1,200 meters long transfer rate of 9,600 bps. This system uses checking and warning the abnormal situation. More over this system has ability to control switch on/off the electrical equipment in the house via AC line control system. The system consists of 4 parts. The first part is a security system of each house using MCS-51 microcontroller as a central processing unit scan 32 sensors and control 8 appliances and send alarm. The MCS-51 microcontroller received control signal via telephone used DTMF circuit. The second part is distributed two levels master/slave network implementing after RS-485 serial communication standard. The protocol its base on the OSI (Open Systems Interconnection) 7 layers protocol model design focus on speed, reliability and security of data that is transferred. The network security using encrypt by DES algorithm, message sequence, time stamp checking and authentication system when user to access and when connect new device to this system. Flow control in system is Poll/Select and Stop-and-Wait method. The third part is central server that using microcomputer which its main function are storing event data into database and can check history event. The final part is internet system which users can access their own homes via the Internet. This web service is based on a combination of SOAP, HTTP and TCP/IP protocols. Messages are exchanged using XML format [6]. In order to save the number of IP address, the system uses 1 IP address for the whole village in which all homes and appliance in this village are addressed using internal identification numbers. This proposed system gives the data transfer accuracy over 99.8% and maximum polling time is 1,120 ms.

  • PDF

Data-Driven Signal Decomposition using Improved Ensemble EMD Method (개선된 앙상블 EMD 방법을 이용한 데이터 기반 신호 분해)

  • Lee, Geum-Boon
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.19 no.2
    • /
    • pp.279-286
    • /
    • 2015
  • EMD is a fully data-driven signal processing method without using any predetermined basis function and requiring any user parameters setting. However EMD experiences a problem of mode mixing which interferes with decomposing the signal into similar oscillations within a mode. To overcome the problem, EEMD method was introduced. The algorithm performs the EMD method over an ensemble of the signal added independent identically distributed white noise of the same standard deviation. Even so EEMD created problems when the decomposition is complete. The ensemble of different signal with added noise may produce different number of modes and the reconstructed signal includes residual noise. This paper propose an modified EEMD method to overcome mode mixing of EMD, to provide an exact reconstruction of the original signal, and to separate modes with lower cost than EEMD's. The experimental results show that the proposed method provides a better separation of the modes with less number of sifting iterations, costs 20.87% for a complete decomposition of the signal and demonstrates superior performance in the signal reconstruction, compared with EEMD.

Long-term Location Data Management for Distributed Moving Object Databases (분산 이동 객체 데이타베이스를 위한 과거 위치 정보 관리)

  • Lee, Ho;Lee, Joon-Woo;Park, Seung-Yong;Lee, Chung-Woo;Hwang, Jae-Il;Nah, Yun-Mook
    • Journal of Korea Spatial Information System Society
    • /
    • v.8 no.2 s.17
    • /
    • pp.91-107
    • /
    • 2006
  • To handling the extreme situation that must manage positional information of a very large volume, at least millions of moving objects. A cluster-based sealable distributed computing system architecture, called the GALIS which consists of multiple data processors, each dedicated to keeping records relevant to a different geographical zone and a different time zone, was proposed. In this paper, we proposed a valid time management and time-zone shifting scheme, which are essential in realizing the long-term location data subsystem of GALIS, but missed in our previous prototype development. We explain how to manage valid time of moving objects to avoid ambiguity of location information. We also describe time-zone shifting algorithm with three variations, such as Real Time-Time Zone Shifting, Batch-Time Zone Shifting, Table Partitioned Batch-Time Zone Shifting, Through experiments related with query processing time and CPU utilization, we show the efficiency of the proposed time-zone shifting schemes.

  • PDF

Digital image watermarking techniques using multiresolution wavelet transform in Sequency domain (다해상도 웨이브렛 변환을 사용한 주파수 영역에서의 디지털 영상 워터마킹 기법)

  • 신종홍;연현숙;지인호
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.26 no.12A
    • /
    • pp.2074-2084
    • /
    • 2001
  • la this paper, a new digital watermarking algorithm using wavelet transform in frequency domain is suggested. The wavelet coefficients of low frequency subband are utilized to embed the watermark, After the original image is transformed using discrete wavelet transform, their coefficients are transformed into efficient1y in Sequency domain. DCT and FFT transforms are utilized in this processing. Watermark image of general image format is transformed using DCT and the hiding watermark into wavelet coefficients is equally distributed in frequency domain. Next, these wavelet coefficients are performed with inverse transform. The detection process of watermark is performed with reverse direction to insertion process. In this paper, we developed core watermark technologies which are a data hiding technology to hide unique logo mark which symbolizes the copyright and a robust protection technology to protect logo data from external attack like as compression, filtering, resampling, cropping. The experimental results show that two suggested watermarking technologies are invisible and robust.

  • PDF

A modified U-net for crack segmentation by Self-Attention-Self-Adaption neuron and random elastic deformation

  • Zhao, Jin;Hu, Fangqiao;Qiao, Weidong;Zhai, Weida;Xu, Yang;Bao, Yuequan;Li, Hui
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.1-16
    • /
    • 2022
  • Despite recent breakthroughs in deep learning and computer vision fields, the pixel-wise identification of tiny objects in high-resolution images with complex disturbances remains challenging. This study proposes a modified U-net for tiny crack segmentation in real-world steel-box-girder bridges. The modified U-net adopts the common U-net framework and a novel Self-Attention-Self-Adaption (SASA) neuron as the fundamental computing element. The Self-Attention module applies softmax and gate operations to obtain the attention vector. It enables the neuron to focus on the most significant receptive fields when processing large-scale feature maps. The Self-Adaption module consists of a multiplayer perceptron subnet and achieves deeper feature extraction inside a single neuron. For data augmentation, a grid-based crack random elastic deformation (CRED) algorithm is designed to enrich the diversities and irregular shapes of distributed cracks. Grid-based uniform control nodes are first set on both input images and binary labels, random offsets are then employed on these control nodes, and bilinear interpolation is performed for the rest pixels. The proposed SASA neuron and CRED algorithm are simultaneously deployed to train the modified U-net. 200 raw images with a high resolution of 4928 × 3264 are collected, 160 for training and the rest 40 for the test. 512 × 512 patches are generated from the original images by a sliding window with an overlap of 256 as inputs. Results show that the average IoU between the recognized and ground-truth cracks reaches 0.409, which is 29.8% higher than the regular U-net. A five-fold cross-validation study is performed to verify that the proposed method is robust to different training and test images. Ablation experiments further demonstrate the effectiveness of the proposed SASA neuron and CRED algorithm. Promotions of the average IoU individually utilizing the SASA and CRED module add up to the final promotion of the full model, indicating that the SASA and CRED modules contribute to the different stages of model and data in the training process.

An Adaptive Chord for Minimizing Network Traffic in a Mobile P2P Environment (비정기적 데이터 수집 모드에 기반한 효율적인 홈 네트워크 서비스 제어 시스템의 설계)

  • Woo, Hyun-Je;Lee, Mee-Jeong
    • The KIPS Transactions:PartC
    • /
    • v.16C no.6
    • /
    • pp.773-782
    • /
    • 2009
  • A DHT(Distributed Hash Table) based P2P is a method to overcome disadvantages of the existing unstructured P2P method. If a DHT algorithm is used, it can do a fast data search and maintain search efficiency independent of the number of peer. The peers in the DHT method send messages periodically to keep the routing table updated. In a mobile environment, the peers in the DHT method should send messages more frequently to keep the routing table updated and reduce the failure of a request. Therefore, this results in increase of network traffic. In our previous research, we proposed a method to reduce the update load of the routing table in the existing Chord by updating it in a reactive way, but the reactive method had a disadvantage to generate more traffic than the existing Chord if the number of requests per second becomes large. In this paper, we propose an adaptive method of routing table update to reduce the network traffic. In the proposed method, we apply different routing table update method according to the number of request message per second. If the number of request message per second is smaller than some threshold, we apply the reactive method. Otherwsie, we apply the existing Chord method. We perform experiments using Chord simulator (I3) made by UC Berkeley. The experimental results show the performance improvement of the proposed method compared to the existing methods.

An Adaptive Chord for Minimizing Network Traffic in a Mobile P2P Environment (모바일 P2P 환경에서 네트워크 트래픽을 최소화한 적응적인 Chord)

  • Yoon, Young-Hyo;Kwak, Hu-Keun;Kim, Cheong-Ghil;Chung, Kyu-Sik
    • The KIPS Transactions:PartC
    • /
    • v.16C no.6
    • /
    • pp.761-772
    • /
    • 2009
  • A DHT(Distributed Hash Table) based P2P is a method to overcome disadvantages of the existing unstructured P2P method. If a DHT algorithm is used, it can do a fast data search and maintain search efficiency independent of the number of peer. The peers in the DHT method send messages periodically to keep the routing table updated. In a mobile environment, the peers in the DHT method should send messages more frequently to keep the routing table updated and reduce the failure of a request. Therefore, this results in increase of network traffic. In our previous research, we proposed a method to reduce the update load of the routing table in the existing Chord by updating it in a reactive way, but the reactive method had a disadvantage to generate more traffic than the existing Chord if the number of requests per second becomes large. In this paper, we propose an adaptive method of routing table update to reduce the network traffic. In the proposed method, we apply different routing table update method according to the number of request message per second. If the number of request message per second is smaller than some threshold, we apply the reactive method. Otherwsie, we apply the existing Chord method. We perform experiments using Chord simulator (I3) made by UC Berkeley. The experimental results show the performance improvement of the proposed method compared to the existing methods.

Cloud P2P OLAP: Query Processing Method and Index structure for Peer-to-Peer OLAP on Cloud Computing (Cloud P2P OLAP: 클라우드 컴퓨팅 환경에서의 Peer-to-Peer OLAP 질의처리기법 및 인덱스 구조)

  • Joo, Kil-Hong;Kim, Hun-Dong;Lee, Won-Suk
    • Journal of Internet Computing and Services
    • /
    • v.12 no.4
    • /
    • pp.157-172
    • /
    • 2011
  • The latest active studies on distributed OLAP to adopt a distributed environment are mainly focused on DHT P2P OLAP and Grid OLAP. However, these approaches have its weak points, the P2P OLAP has limitations to multidimensional range queries in the cloud computing environment due to the nature of structured P2P. On the other hand, the Grid OLAP has no regard for adjacency and time series. It focused on its own sub set lookup algorithm. To overcome the above limits, this paper proposes an efficient central managed P2P approach for a cloud computing environment. When a multi-level hybrid P2P method is combined with an index load distribution scheme, the performance of a multi-dimensional range query is enhanced. The proposed scheme makes the OLAP query results of a user to be able to reused by other users' volatile cube search. For this purpose, this paper examines the combination of an aggregation cube hierarchy tree, a quad-tree, and an interval-tree as an efficient index structure. As a result, the proposed cloud P2P OLAP scheme can manage the adjacency and time series factor of an OLAP query. The performance of the proposed scheme is analyzed by a series of experiments to identify its various characteristics.

Investigating Data Preprocessing Algorithms of a Deep Learning Postprocessing Model for the Improvement of Sub-Seasonal to Seasonal Climate Predictions (계절내-계절 기후예측의 딥러닝 기반 후보정을 위한 입력자료 전처리 기법 평가)

  • Uran Chung;Jinyoung Rhee;Miae Kim;Soo-Jin Sohn
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.25 no.2
    • /
    • pp.80-98
    • /
    • 2023
  • This study explores the effectiveness of various data preprocessing algorithms for improving subseasonal to seasonal (S2S) climate predictions from six climate forecast models and their Multi-Model Ensemble (MME) using a deep learning-based postprocessing model. A pipeline of data transformation algorithms was constructed to convert raw S2S prediction data into the training data processed with several statistical distribution. A dimensionality reduction algorithm for selecting features through rankings of correlation coefficients between the observed and the input data. The training model in the study was designed with TimeDistributed wrapper applied to all convolutional layers of U-Net: The TimeDistributed wrapper allows a U-Net convolutional layer to be directly applied to 5-dimensional time series data while maintaining the time axis of data, but every input should be at least 3D in U-Net. We found that Robust and Standard transformation algorithms are most suitable for improving S2S predictions. The dimensionality reduction based on feature selections did not significantly improve predictions of daily precipitation for six climate models and even worsened predictions of daily maximum and minimum temperatures. While deep learning-based postprocessing was also improved MME S2S precipitation predictions, it did not have a significant effect on temperature predictions, particularly for the lead time of weeks 1 and 2. Further research is needed to develop an optimal deep learning model for improving S2S temperature predictions by testing various models and parameters.