• Title/Summary/Keyword: Cluster Computing

Search Result 425, Processing Time 0.028 seconds

A Reservation based Network Resource Provisioning Testbed Using the Integrated Resource Management System (통합자원관리시스템을 이용한 예약 기반의 네트워크 자원 할당 테스트베드 망)

  • Lim, Huhn-Kuk;Moon, Jeong-Hoon;Kong, Jong-Uk;Han, Jang-Soo;Cha, Young-Wook
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.36 no.12B
    • /
    • pp.1450-1458
    • /
    • 2011
  • The HPcN (Hybrid & high Performance Convergence Network) in research networks means environment which can provide both computing resource such as supercomputer, cluster and network resource to application researchers in the field of medical, bio, aerospace and e-science. The most representative research network in Korea, KREONET has been developing following technologies through the HERO (Hybrid Networking project for research oriented infrastructure) from 200S. First, we have constructed and deployed a control plane technology which can provide a connection oriented network dynamically. Second, the integrated resource management system technology has been developing for reservation and allocation of both computing and network resources, whenever users want to utilize them. In this paper, a testbed network is presented, which is possible to reserve and allocate network resource using the integrated resource management system. We reserve network resource through GNSI (Grid Network Service Interface) messages between GRS (Global Resource Scheduler) and NRM (Network Resource Manager) and allocate network resource through GUNI (Grid User Network Interface) messages between the NRM (network resource manager) and routers, based on reservation information provided from a user on the web portal. It is confirmed that GUNI interface messages are delivered from the NRM to each router at the starting of reservation time and traffic is transmitted through LSP allocated by the NRM.

Broadcasting and Caching Schemes for Location-dependent Queries in Urban Areas (도심환경에서 위치의존 질의를 위한 방송과 캐싱 기법)

  • Jung Il-dong;Yu Young-ho;Lee Jong-hwan;Kim Kyongsok
    • Journal of KIISE:Databases
    • /
    • v.32 no.1
    • /
    • pp.56-70
    • /
    • 2005
  • The results of location-dependent queries(LDQ) generally depend on the current locations of query issuers. Many mechanisms, e.g. broadcast scheme, hoarding, or racking policy, have been developed to improve the system peformance and provide better services, which are specialized for LDQs. Considering geographical adjacency of data and characteristics oi target area, caching policy and broadcast scheme affect the overall performance in LDQ. For this reason, we propose both the caching policy and broadcast scheme, which these features are reflected in. Based on the adjacency of data in LDQ, our broadcast scheme use Hilbert curve to cluster data. Moreover, in order to develop the caching policy suitable for LDQ on urban area, we apply the moving distance of a MH(Mobile Host) to our caching policy We evaluate the performance of the caching policy measuring the workload of MHs and the correctness of LDQ results and the performance of the broadcast scheme measuring the average setup-time of MHs in our experiments. Finally, we expect that our caching policy Provides more correct answers when executing LDQ in focal cache and leads significant improvement of the performance of MHs. It also seems quite probable that our broadcast scheme leads improvement of battery life of the MH.

Development of Retargetable Hadoop Simulation Environment Based on DEVS Formalism (DEVS 형식론 기반의 재겨냥성 하둡 시뮬레이션 환경 개발)

  • Kim, Byeong Soo;Kang, Bong Gu;Kim, Tag Gon;Song, Hae Sang
    • Journal of the Korea Society for Simulation
    • /
    • v.26 no.4
    • /
    • pp.51-61
    • /
    • 2017
  • Hadoop platform is a representative storing and managing platform for big data. Hadoop consists of distributed computing system called MapReduce and distributed file system called HDFS. It is important to analyse the effectiveness according to the change of cluster constructions and several parameters. However, since it is hard to construct thousands of clusters and analyse the constructed system, simulation method is required to analyse the system. This paper proposes Hadoop simulator based on DEVS formalism which provides hierarchical and modular modeling. Hadoop simulator provides a retargetable experimental environment that is possible to change of various parameters, algorithms and models. It is also possible to design input models reflecting the characteristics of Hadoop applications. To maximize the user's convenience, the user interface, real-time model viewer, and input scenario editor are also provided. In this paper, we validate Hadoop Simulator through the comparison with the Hadoop execution results and perform various experiments.

Assessment through Statistical Methods of Water Quality Parameters(WQPs) in the Han River in Korea

  • Kim, Jae Hyoun
    • Journal of Environmental Health Sciences
    • /
    • v.41 no.2
    • /
    • pp.90-101
    • /
    • 2015
  • Objective: This study was conducted to develop a chemical oxygen demand (COD) regression model using water quality monitoring data (January, 2014) obtained from the Han River auto-monitoring stations. Methods: Surface water quality data at 198 sampling stations along the six major areas were assembled and analyzed to determine the spatial distribution and clustering of monitoring stations based on 18 WQPs and regression modeling using selected parameters. Statistical techniques, including combined genetic algorithm-multiple linear regression (GA-MLR), cluster analysis (CA) and principal component analysis (PCA) were used to build a COD model using water quality data. Results: A best GA-MLR model facilitated computing the WQPs for a 5-descriptor COD model with satisfactory statistical results ($r^2=92.64$,$Q{^2}_{LOO}=91.45$,$Q{^2}_{Ext}=88.17$). This approach includes variable selection of the WQPs in order to find the most important factors affecting water quality. Additionally, ordination techniques like PCA and CA were used to classify monitoring stations. The biplot based on the first two principal components (PCs) of the PCA model identified three distinct groups of stations, but also differs with respect to the correlation with WQPs, which enables better interpretation of the water quality characteristics at particular stations as of January 2014. Conclusion: This data analysis procedure appears to provide an efficient means of modelling water quality by interpreting and defining its most essential variables, such as TOC and BOD. The water parameters selected in a COD model as most important in contributing to environmental health and water pollution can be utilized for the application of water quality management strategies. At present, the river is under threat of anthropogenic disturbances during festival periods, especially at upstream areas.

A Study On Recommend System Using Co-occurrence Matrix and Hadoop Distribution Processing (동시발생 행렬과 하둡 분산처리를 이용한 추천시스템에 관한 연구)

  • Kim, Chang-Bok;Chung, Jae-Pil
    • Journal of Advanced Navigation Technology
    • /
    • v.18 no.5
    • /
    • pp.468-475
    • /
    • 2014
  • The recommend system is getting more difficult real time recommend by lager preference data set, computing power and recommend algorithm. For this reason, recommend system is proceeding actively one's studies toward distribute processing method of large preference data set. This paper studied distribute processing method of large preference data set using hadoop distribute processing platform and mahout machine learning library. The recommend algorithm is used Co-occurrence Matrix similar to item Collaborative Filtering. The Co-occurrence Matrix can do distribute processing by many node of hadoop cluster, and it needs many computation scale but can reduce computation scale by distribute processing. This paper has simplified distribute processing of co-occurrence matrix by changes over from four stage to three stage. As a result, this paper can reduce mapreduce job and can generate recommend file. And it has a fast processing speed, and reduce map output data.

A Study on an Algorithm for Typical Meteorological Year Generation for Wind Resource of the Korean Peninsula (한반도 바람자원의 TMY(typical meteorological year)구축 알고리즘에 관한 연구)

  • Kim, Hea-Jung;Jung, Sun;Choi, Yeoung-Jin;Kim, Kyu-Rang;Jung, Young-Rim
    • The Korean Journal of Applied Statistics
    • /
    • v.22 no.5
    • /
    • pp.943-960
    • /
    • 2009
  • This study suggests an algorithm for generating TMY(typical meteorological year) for the Korean peninsula, and generates the TMY based on the algorithm using 11 years(1998~2008) wind data observed at 77 sites of Regional Meteorological Offices(RMO). The algorithm consists of computing TMM scores based on the various statistics defined by the Fikenstein-Shafer statistical model and, in turn, generating TMY based on the TMM scores. Also the algorithm has two stages designed to yield the best representation of the regional wind characteristics appeared during the 11 years(1998~2008). The first stage is designed for the representation of each of 77 regions of RMO and the second is for the Korean peninsula. Various comparison studies are provided to demonstrate the properties of the TMY like its utility and typicality.

Implementation of a Large-scale Web Query Processing System Using the Multi-level Cache Scheme (계층적 캐시 기법을 이용한 대용량 웹 검색 질의 처리 시스템의 구현)

  • Lim, Sung-Chae
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.14 no.7
    • /
    • pp.669-679
    • /
    • 2008
  • With the increasing demands of information sharing and searches via the web, the web search engine has drawn much attention. Although many researches have been done to solve technical challenges to build the web search engine, the issue regarding its query processing system is rarely dealt with. Since the software architecture and operational schemes of the query processing system are hard to elaborate, we here present related techniques implemented on a commercial system. The implemented system is a very large-scale system that can process 5-million user queries per day by using index files built on about 65-million web pages. We implement a multi-level cache scheme to save already returned query results for performance considerations, and the multi-level cache is managed in 4-level cache storage areas. Using the multi-level cache, we can improve the system throughput by a factor of 4, thereby reducing around 70% of the server cost.

Power Management Strategy and Performance Evaluation for OpenStack Object Storage (오픈스택 기반 객체 스토리지를 위한 전력관리 기법과 성능 평가)

  • Ahn, Cheong-Jin;Song, Tae-Gun;Lee, Byeong-Hyeon;Kim, Deok-Hwan
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.6
    • /
    • pp.296-301
    • /
    • 2016
  • Object-based storage is an efficient storage solution that can handle unstructured data and shows better security and scalability than traditional block-based storage. However, in terms of power management, Object-based storage writes multiple copies in storage cluster, hence many servers consume unnecessary power in idle state. In order to resolve this problem, it is necessary to apply power management strategy by adjusting power mode of servers in idle state according to their workloads. In this paper, we proposed a new dynamic power management (DPM) method to transform power mode of storage servers dynamically according to workload information sent from proxy server. The experimental result shows that the proposed power management technic reduces total power consumption by 12% in the OpenStack based Swift object storage.

Uncertainty for Privacy and 2-Dimensional Range Query Distortion

  • Sioutas, Spyros;Magkos, Emmanouil;Karydis, Ioannis;Verykios, Vassilios S.
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.3
    • /
    • pp.210-222
    • /
    • 2011
  • In this work, we study the problem of privacy-preservation data publishing in moving objects databases. In particular, the trajectory of a mobile user in a plane is no longer a polyline in a two-dimensional space, instead it is a two-dimensional surface of fixed width $2A_{min}$, where $A_{min}$ defines the semi-diameter of the minimum spatial circular extent that must replace the real location of the mobile user on the XY-plane, in the anonymized (kNN) request. The desired anonymity is not achieved and the entire system becomes vulnerable to attackers, since a malicious attacker can observe that during the time, many of the neighbors' ids change, except for a small number of users. Thus, we reinforce the privacy model by clustering the mobile users according to their motion patterns in (u, ${\theta}$) plane, where u and ${\theta}$ define the velocity measure and the motion direction (angle) respectively. In this case, the anonymized (kNN) request looks up neighbors, who belong to the same cluster with the mobile requester in (u, ${\theta}$) space: Thus, we know that the trajectory of the k-anonymous mobile user is within this surface, but we do not know exactly where. We transform the surface's boundary poly-lines to dual points and we focus on the information distortion introduced by this space translation. We develop a set of efficient spatiotemporal access methods and we experimentally measure the impact of information distortion by comparing the performance results of the same spatiotemporal range queries executed on the original database and on the anonymized one.

The Routing Algorithm for Wireless Sensor Networks with Random Mobile Nodes

  • Yun, Dai Yeol;Jung, Kye-Dong;Lee, Jong-Yong
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.9 no.4
    • /
    • pp.38-43
    • /
    • 2017
  • Sensor Networks (WSNs) can be defined as a self-configured and infrastructure-less wireless networks to monitor physical or environmental conditions, such as temperature, sound, vibration, pressure, motion or pollutants and to cooperatively pass their data through the network to a main location or base-station where the data can be observed and analyzed. Typically a wireless sensor network contains hundreds of thousands of sensor nodes. The sensor nodes can communicate among themselves using radio signals. A wireless sensor node is equipped with sensing and computing devices, radio transceivers and power components. The individual nodes in a wireless sensor network (WSN) are inherently resource constrained: they have limited processing speed, storage capacity, communication bandwidth and limited-battery power. At present time, most of the research on WSNs has concentrated on the design of energy- and computationally efficient algorithms and protocols In order to extend the network life-time, in this paper we are looking into a routing protocol, especially LEACH and LEACH-related protocol. LEACH protocol is a representative routing protocol and improves overall network energy efficiency by allowing all nodes to be selected to the cluster head evenly once in a periodic manner. In LEACH, in case of movement of sensor nodes, there is a problem that the data transmission success rate decreases. In order to overcome LEACH's nodes movements, LEACH-Mobile protocol had proposed. But energy consumption increased because it consumes more energy to recognize which nodes moves and re-transfer data. In this paper we propose the new routing protocol considering nodes' mobility. In order to simulate the proposed protocol, we make a scenario, nodes' movements randomly and compared with the LEACH-Mobile protocol.