• Title/Summary/Keyword: Batch processing

Search Result 292, Processing Time 0.029 seconds

An Improved Skyline Query Scheme for Recommending Real-Time User Preference Data Based on Big Data Preprocessing (빅데이터 전처리 기반의 실시간 사용자 선호 데이터 추천을 위한 개선된 스카이라인 질의 기법)

  • Kim, JiHyun;Kim, Jongwan
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.5
    • /
    • pp.189-196
    • /
    • 2022
  • Skyline query is a scheme for exploring objects that are suitable for user preferences based on multiple attributes of objects. Existing skyline queries return search results as batch processing, but the need for real-time search results has increased with the advent of interactive apps or mobile environments. Online algorithm for Skyline improves the return speed of objects to explore preferred objects in real time. However, the object navigation process requires unnecessary navigation time due to repeated comparative operations. This paper proposes a Pre-processing Online Algorithm for Skyline Query (POA) to eliminate unnecessary search time in Online Algorithm exploration techniques and provide the results of skyline queries in real time. Proposed techniques use the concept of range-limiting to existing Online Algorithm to perform pretreatment and then eliminate repetitive rediscovering regions first. POAs showed improvement in standard distributions, bias distributions, positive correlations, and negative correlations of discrete data sets compared to Online Algorithm. The POAs used in this paper improve navigation performance by minimizing comparison targets for Online Algorithm, which will be a new criterion for rapid service to users in the face of increasing use of mobile devices.

Design of Trajectory Data Indexing and Query Processing for Real-Time LBS in MapReduce Environments (MapReduce 환경에서의 실시간 LBS를 위한 이동궤적 데이터 색인 및 검색 시스템 설계)

  • Chung, Jaehwa
    • Journal of Digital Contents Society
    • /
    • v.14 no.3
    • /
    • pp.313-321
    • /
    • 2013
  • In recent, proliferation of mobile smart devices have led to big-data era, the importance of location-based services is increasing due to the exponential growth of trajectory related data. In order to process trajectory data, parallel processing platforms such as cloud computing and MapReduce are necessary. Currently, the researches based on MapReduce are on progress, but due to the MapReduce's properties in using batch processing and simple key-value structure, applying MapReduce framework for real time LBS is difficult. Therefore, in this research we propose a suitable system design on efficient indexing and search techniques for real time service based on detailed analysis on the properties of MapReduce.

Dynamic Load Management Method for Spatial Data Stream Processing on MapReduce Online Frameworks (맵리듀스 온라인 프레임워크에서 공간 데이터 스트림 처리를 위한 동적 부하 관리 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.8
    • /
    • pp.535-544
    • /
    • 2018
  • As the spread of mobile devices equipped with various sensors and high-quality wireless network communications functionsexpands, the amount of spatio-temporal data generated from mobile devices in various service fields is rapidly increasing. In conventional research into processing a large amount of real-time spatio-temporal streams, it is very difficult to apply a Hadoop-based spatial big data system, designed to be a batch processing platform, to a real-time service for spatio-temporal data streams. This paper extends the MapReduce online framework to support real-time query processing for continuous-input, spatio-temporal data streams, and proposes a load management method to distribute overloads for efficient query processing. The proposed scheme shows a dynamic load balancing method for the nodes based on the inflow rate and the load factor of the input data based on the space partition. Experiments show that it is possible to support efficient query processing by distributing the spatial data stream in the corresponding area to the shared resources when load management in a specific area is required.

Biodiesel Production: Utilization of Loofah Sponge to Immobilize Rhizopus chinensis CGMCC #3.0232 Cells as a Whole-Cell Biocatalyst

  • He, Qiyang;Xia, Qianjun;Wang, Yuejiao;Li, Xun;Zhang, Yu;Hu, Bo;Wang, Fei
    • Journal of Microbiology and Biotechnology
    • /
    • v.26 no.7
    • /
    • pp.1278-1284
    • /
    • 2016
  • Rhizopus chinensis cells immobilized on loofah (Luffa cylindrica) sponges were used to produce biodiesel via the transesterification of soybean oil. In whole-cell immobilization, loofah sponge is considered to be a superior alternative to conventional biomass carriers because of its biodegradable and renewable properties. During cell cultivation, Rhizopus chinensis mycelia can spontaneously and firmly adhere to the surface of loofah sponge particles. The optimal conditions for processing 9.65 g soybean oil at 40℃ and 180 rpm using a 3:1 methanol-to-oil molar ratio were found to be 8% cell addition and 3-10% water content (depending on the oil's weight). Under optimal conditions, an over 90% methyl ester yield was achieved after the first reaction batch. The operational stability of immobilized Rhizopus chinensis cells was assayed utilizing a 1:1 methanol-to-oil molar ratio, thus resulting in a 16.5-fold increase in half-life when compared with immobilized cells of the widely studied Rhizopus oryzae. These results suggest that transesterification of vegetable oil using Rhizopus chinensis whole cells immobilized onto loofah sponge is an effective approach for biodiesel production.

Upcycling of Waste Jelly-Filled Communication Cables (폐 젤리충진 통신케이블 업사이클링 연구)

  • Cho, Sungsu;Lee, Sooyoung;Hong, Myunghwan;Seo, Minhye;Lee, Dukhee;Uhm, Sunghyun
    • Resources Recycling
    • /
    • v.24 no.2
    • /
    • pp.29-35
    • /
    • 2015
  • A feasibility test was carried out for upcycling of waste jelly-filled communication cables together with the development of environmentally friendly processes and equipments. High pressure water injection is proved to be an exceptionally environmentally friendly and highly efficient mechanical process. A batch-type cable barking equipment is designed and built on the basis of computational fluid dynamics modelling. It is optimized in terms of energy consumption and productivity with very high copper recovery of 99.5%. Copper nano-powder is prepared by an electrical wire explosion in ethanol media in order to improve the value of final products, and the preliminary economical assessment is also conducted.

Handling Streaming Data by Using Open Source Framework Storm in IoT Environment (오픈소스 프레임워크 Storm을 활용한 IoT 환경 스트리밍 데이터 처리)

  • Kang, Yunhee
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.7
    • /
    • pp.313-318
    • /
    • 2016
  • To utilize sensory data, it is necessary to design architecture for processing and handling data generated from sensors in an IoT environment. Especially in the IoT environment, a thing connects to the Internet and efficiently enables to communicate a device with diverse sensors. But Hadoop and Twister based on MapReduce are good at handling data in a batch processing. It has a limitation for processing stream data from a sensor in a motion. Traditional streaming data processing has been mainly applied a MoM based message queuing system. It has maintainability and scalability problems because a programmer should consider details related with complex messaging flow. In this paper architecture is designed to handle sensory data aggregated The designed software architecture is used to operate an application on the open source framework Storm. The application is conceptually used to transform streaming data which aggregated via sensor gateway by pipe-filter style.

Treatment of Photographic Wastewater by Chemical Oxidation and Biological Treatment process (화학적산화 및 생물학적처리법에 의한 사진폐액의 처리)

  • 정경훈;최형일
    • Journal of Environmental Health Sciences
    • /
    • v.23 no.1
    • /
    • pp.34-42
    • /
    • 1997
  • A laboratory experiments were performed to investigate the treatment of photographic processing wastewater by chemical oxidation and biological treatment system. The effect of reaction conditions such as hydrogen peroxide dosage, ferrous sulfate dosage and pH on the COD removal in Fenton oxidation were investigated. The optimal dosage of hydrogen peroxide was 2.58 M and 3.87 M for the developing and fixing process wastewater, respectively. The Fenton oxidation was most efficient in the pH range of 3-5 and the optimal condition for initial reaction pH was 5 for a developing process wastewater. With iron powder catalyst, the COD for a developing process wastewater was removed in lower pH than with ferrous sulfate catalyst. The removal efficiency of COD for refractory compounds such as Diethyleneglycol, Benzylalcohol, Hydroxylamine Sulfate, Ammonium Thiosulfate, Ammonium Ferric EDTA and Disodium EDTA in the photogaphic wastewater was found than 90% except Potassium Carbonate. When the photographic processing wastewater after pretreatment by Fenton oxidation was treated with batch activated sludge process, the addition of $KH_2PO_4$ as a phosphorous compound improved the removal efficiency of COD. During the continuous biological treatment of developing and fixing process wastewater after pretreatment by Fenton oxidation, the effluent COD concentration less than 100 mg/l was obtained at 0.425 and 0.25 kgCOD/m$^3$.d, respectively.

  • PDF

Yeast Single-Cell Protein Production Using Potato Processing Waste Water

  • Park, Eung-Yeal;Crawford, Don-L.;Korus, Roger-A.;Heimsch, Richard-D.
    • Journal of Microbiology and Biotechnology
    • /
    • v.1 no.3
    • /
    • pp.212-219
    • /
    • 1991
  • Four species of yeast, Saccharomyces cerevisiae, Candida utilis, Saccharomycopsis flbuligera, and Schwanniomyces castellii were evaluated for their ability to bioconvert potato processing waste water into microbial protein and the resulting single-cell proteins were evaluated as protein sources for rainbow trout, using in vitro analyses. The studies indicated that Schwanniomyces castellii, which utilizes starch dircetly and converts it into cell mass efficiently, was suitable for the bioconversion. In the single-stage continuous bioconversion, the yield S. castellii cell mass, which contained approximately 37% protein, was 77%, at dilution rate 0.25 $h^{-1}$. Reduction of total carbohydrate was 81%. During batch fermentations, cell mass yield was about 72% and total carbohydrate reduction was 81%. Among the yeasts tested, S. castellii possessed the most fragile cell wall and had a favorable amino acid profile for salmonid fish; protein score of 86% (Met). In an in vitro pepsin digestibility test 80% digestibility (23~38% above control) was observed when cells were pre-heated in a steam bath for 30 min. Results presented should be regarded as being preliminary in nature because they were derived from single experiments.

  • PDF

Mixed-Integer programming model for scheduling of steelmaking processes (철강 공정의 일정계획을 위한 혼합정수계획 모델)

  • Bok, Jin-Gwang;Lee, Dong-Yeop;Park, Seon-Won
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.5 no.6
    • /
    • pp.714-723
    • /
    • 1999
  • This paper presents a short-term scheduling algorithm for the operation of steelmaking processes. The scope of the problem covers refining of the hot iron transferred form a blast furnace, ladle treatment, continuous casting, hot-rolling, and coiling for the final products that should satisfy the given demand. The processing time at each unit depends on how much the batch amount is treated, and te dedicated intermediate storage with finite capacity between the units is considered. Resource constraints and initial amount of each state are incorporated into the presented scheduling model for the algorithm of on-line scheduling. We propose amixed integer linear programming (MILP) model with two objectives for the scheduling. The first is to maximize the total profit while atisfying the due date constraint for each product. And the second is to minimize the total processing time, makespan, while satisfying the demand for each product. Especially, we observe the effect of penalizing the intermediate storage and the inventory level of the final product on the scheduling results.

  • PDF

Progress in research and development for REBCO coated conductors by reactive co-evaporation

  • Oh, S.S.;Kim, H.S.;Ha, H.S.;Ko, R.K.;Ha, D.W.;Lee, H.;Moon, S.H.;Yoo, S.I.
    • Progress in Superconductivity and Cryogenics
    • /
    • v.15 no.4
    • /
    • pp.1-5
    • /
    • 2013
  • This paper reviews recent progress in research and development (R&D) of reactive co-evaporation for high performance REBCO coated conductors in Korea. Two types of reactive co-evaporation methods were developed for the deposition of SmBCO and GdBCO superconducting layers respectively on the IBAD (Ion Beam Assisted Deposition)-MgO template in the Korean coated conductor project. Batch type reactive co-evaporation equipment and its processing were developed for SmBCO coated conductors at Korea Electrotechnology Research Institute (KERI) in conjunction with the Korea Advanced Institute of Science and Technology (KAIST), and a very high critical current exceeding 1,000 A/cm at 77 K in the self field was achieved through the optimization of deposition parameters. Reel-to-reel type reactive co-evaporation processing with a high conversion rate was also developed, while long length GdBCO coated conductors have been routinely produced by SuNAM Co. The minimum critical current of 422 A/cm-w at 77 K in self field was confirmed for 1 km-long GdBCO tape.