• Title/Summary/Keyword: Data Optimization

Search Result 3,447, Processing Time 0.037 seconds

A Hybrid Mechanism of Particle Swarm Optimization and Differential Evolution Algorithms based on Spark

  • Fan, Debin;Lee, Jaewan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.12
    • /
    • pp.5972-5989
    • /
    • 2019
  • With the onset of the big data age, data is growing exponentially, and the issue of how to optimize large-scale data processing is especially significant. Large-scale global optimization (LSGO) is a research topic with great interest in academia and industry. Spark is a popular cloud computing framework that can cluster large-scale data, and it can effectively support the functions of iterative calculation through resilient distributed datasets (RDD). In this paper, we propose a hybrid mechanism of particle swarm optimization (PSO) and differential evolution (DE) algorithms based on Spark (SparkPSODE). The SparkPSODE algorithm is a parallel algorithm, in which the RDD and island models are employed. The island model is used to divide the global population into several subpopulations, which are applied to reduce the computational time by corresponding to RDD's partitions. To preserve population diversity and avoid premature convergence, the evolutionary strategy of DE is integrated into SparkPSODE. Finally, SparkPSODE is conducted on a set of benchmark problems on LSGO and show that, in comparison with several algorithms, the proposed SparkPSODE algorithm obtains better optimization performance through experimental results.

ON SURROGATE DUALITY FOR ROBUST SEMI-INFINITE OPTIMIZATION PROBLEM

  • Lee, Gue Myung;Lee, Jae Hyoung
    • Journal of the Chungcheong Mathematical Society
    • /
    • v.27 no.3
    • /
    • pp.433-438
    • /
    • 2014
  • A semi-infinite optimization problem involving a quasi-convex objective function and infinitely many convex constraint functions with data uncertainty is considered. A surrogate duality theorem for the semi-infinite optimization problem is given under a closed and convex cone constraint qualification.

A Data Fitting Technique for Rational Function Models Using the LM Optimization Algorithm (LM 최적화 알고리즘을 이용한 유리함수 모델의 데이터 피팅)

  • Park, Jae-Han;Bae, Ji-Hun;Baeg, Moon-Hong
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.17 no.8
    • /
    • pp.768-776
    • /
    • 2011
  • This paper considers a data fitting problem for rational function models using the LM (Levenberg-Marquardt) optimization method. Rational function models have various merits on representing a wide range of shapes and modeling complicated structures by polynomials of low degrees in both the numerator and denominator. However, rational functions are nonlinear in the parameter vector, thereby requiring nonlinear optimization methods to solve the fitting problem. In this paper, we propose a data fitting method for rational function models based on the LM algorithm which is renowned as an effective nonlinear optimization technique. Simulations show that the fitting results are robust against the measurement noises and uncertainties. The effectiveness of the proposed method is further demonstrated by the real application to a 3D depth camera calibration problem.

Developing Novel Algorithms to Reduce the Data Requirements of the Capture Matrix for a Wind Turbine Certification (풍력 발전기 평가를 위한 수집 행렬 데이터 절감 알고리즘 개발)

  • Lee, Jehyun;Choi, Jungchul
    • New & Renewable Energy
    • /
    • v.16 no.1
    • /
    • pp.15-24
    • /
    • 2020
  • For mechanical load testing of wind turbines, capture matrix is constructed for various range of wind speeds according to the international standard IEC 61400-13. The conventional method wastes considerable amount of data by its invalid data policy -segment data into 10 minutes then remove invalid ones. Previously, we have suggested an alternative way to save the total amount of data to build a capture matrix, but the efficient selection of data has been still under question. The paper introduces optimization algorithms to construct capture matrix with less data. Heuristic algorithm (simple stacking and lowest frequency first), population method (particle swarm optimization) and Q-Learning accompanied with epsilon-greedy exploration are compared. All algorithms show better performance than the conventional way, where the distribution of enhancement was quite diverse. Among the algorithms, the best performance was achieved by heuristic method (lowest frequency first), and similarly by particle swarm optimization: Approximately 28% of data reduction in average and more than 40% in maximum. On the other hand, unexpectedly, the worst performance was achieved by Q-Learning, which was a promising candidate at the beginning. This study is helpful for not only wind turbine evaluation particularly the viewpoint of cost, but also understanding nature of wind speed data.

RIS Selection and Energy Efficiency Optimization for Irregular Distributed RIS-assisted Communication Systems

  • Xu Fangmin;Fu Jinzhao;Cao HaiYan;Hu ZhiRui
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.7
    • /
    • pp.1823-1840
    • /
    • 2023
  • In order to improve spectral efficiency and reduce power consumption for reconfigurable intelligent surface (RIS) assisted wireless communication systems, a joint design considering irregular RIS topology, RIS on-off switch, power allocation and phase adjustment is investigated in this paper. Firstly, a multi-dimensional variable joint optimization problem is established under multiple constraints, such as the minimum data requirement and power constraints, with the goal of maximizing the system energy efficiency. However, the proposed optimization problem is hard to be resolved due to its property of nonlinear nonconvex integer programming. Then, to tackle this issue, the problem is decomposed into four sub-problems: topology design, phase shift adjustment, power allocation and switch selection. In terms of topology design, Tabu search algorithm is introduced to select the components that play the main role. For RIS switch selection, greedy algorithm is used to turn off the RISs that play the secondary role. Finally, an iterative optimization algorithm with high data-rate and low power consumption is proposed. The simulation results show that the performance of the irregular RIS aided system with topology design and RIS selection is better than that of the fixed topology and the fix number of RISs. In addition, the proposed joint optimization algorithm can effectively improve the data rate and energy efficiency by changing the propagation environment.

Evolutionary Optimization Design Technique for Control of Solid-Fluid Coupled Force (고체-유체 연성력 제어를 위한 진화적 최적설계)

  • Kim H.S.;Lee Y.S.
    • Proceedings of the Korean Society of Precision Engineering Conference
    • /
    • 2005.06a
    • /
    • pp.503-506
    • /
    • 2005
  • In this study, optimization design technique for control of solid-fluid coupled force (sloshing) using evolutionary method is suggested. Artificial neural networks(ANN) and genetic algorithm(GA) is employed as evolutionary optimization method. The ANN is used to analysis of the sloshing and the genetic algorithm is adopted as an optimization algorithm. In the creation of ANN learning data, the design of experiments is adopted to higher performance of the ANN learning using minimum learning data and ALE(Arbitrary Lagrangian Eulerian) numerical method is used to obtain the sloshing analysis results. The proposed optimization technique is applied to the minimization of sloshing of the water in the tank lorry with baffles under 2 second lane change.

  • PDF

Structural damage identification using cloud model based fruit fly optimization algorithm

  • Zheng, Tongyi;Liu, Jike;Luo, Weili;Lu, Zhongrong
    • Structural Engineering and Mechanics
    • /
    • v.67 no.3
    • /
    • pp.245-254
    • /
    • 2018
  • In this paper, a Cloud Model based Fruit Fly Optimization Algorithm (CMFOA) is presented for structural damage identification, which is a global optimization algorithm inspired by the foraging behavior of fruit fly swarm. It is assumed that damage only leads to the decrease in elementary stiffness. The differences on time-domain structural acceleration data are used to construct the objective function, which transforms the damaged identification problem of a structure into an optimization problem. The effectiveness, efficiency and accuracy of the CMFOA are demonstrated by two different numerical simulation structures, including a simply supported beam and a cantilevered plate. Numerical results show that the CMFOA has a better capacity for structural damage identification than the basic Fruit Fly Optimization Algorithm (FOA) and the CMFOA is not sensitive to measurement noise.

Optimizing Food Processing through a New Approach to Response Surface Methodology

  • Sungsue Rheem
    • Food Science of Animal Resources
    • /
    • v.43 no.2
    • /
    • pp.374-381
    • /
    • 2023
  • In a previous study, 'response surface methodology (RSM) using a fullest balanced model' was proposed to improve the optimization of food processing when a standard second-order model has a significant lack of fit. However, that methodology can be used when each factor of the experimental design has five levels. In response surface experiments for optimization, not only five-level designs, but also three-level designs are used. Therefore, the present study aimed to improve the optimization of food processing when the experimental factors have three levels through a new approach to RSM. This approach employs three-step modeling based on a second-order model, a balanced higher-order model, and a balanced highest-order model. The dataset from the experimental data in a three-level, two-factor central composite design in a previous research was used to illustrate three-step modeling and the subsequent optimization. The proposed approach to RSM predicted improved results of optimization, which are different from the predicted optimization results in the previous research.

Query Optimization on Large Scale Nested Data with Service Tree and Frequent Trajectory

  • Wang, Li;Wang, Guodong
    • Journal of Information Processing Systems
    • /
    • v.17 no.1
    • /
    • pp.37-50
    • /
    • 2021
  • Query applications based on nested data, the most commonly used form of data representation on the web, especially precise query, is becoming more extensively used. MapReduce, a distributed architecture with parallel computing power, provides a good solution for big data processing. However, in practical application, query requests are usually concurrent, which causes bottlenecks in server processing. To solve this problem, this paper first combines a column storage structure and an inverted index to build index for nested data on MapReduce. On this basis, this paper puts forward an optimization strategy which combines query execution service tree and frequent sub-query trajectory to reduce the response time of frequent queries and further improve the efficiency of multi-user concurrent queries on large scale nested data. Experiments show that this method greatly improves the efficiency of nested data query.

A novel WOA-based structural damage identification using weighted modal data and flexibility assurance criterion

  • Chen, Zexiang;Yu, Ling
    • Structural Engineering and Mechanics
    • /
    • v.75 no.4
    • /
    • pp.445-454
    • /
    • 2020
  • Structural damage identification (SDI) is a crucial step in structural health monitoring. However, some of the existing SDI methods cannot provide enough identification accuracy and efficiency in practice. A novel whale optimization algorithm (WOA) based method is proposed for SDI by weighting modal data and flexibility assurance criterion in this study. At first, the SDI problem is mathematically converted into a constrained optimization problem. Unlike traditional objective function defined using frequencies and mode shapes, a new objective function on the SDI problem is formulated by weighting both modal data and flexibility assurance criterion. Then, the WOA method, due to its good performance of fast convergence and global searching ability, is adopted to provide an accurate solution to the SDI problem, different predator mechanisms are formulated and their probability thresholds are selected. Finally, the performance of the proposed method is assessed by numerical simulations on a simply-supported beam and a 31-bar truss structures. For the given multiple structural damage conditions under environmental noises, the WOA-based SDI method can effectively locate structural damages and accurately estimate severities of damages. Compared with other optimization methods, such as particle swarm optimization and dragonfly algorithm, the proposed WOA-based method outperforms in accuracy and efficiency, which can provide a more effective and potential tool for the SDI problem.