• Title/Summary/Keyword: least-squares methods

Search Result 624, Processing Time 0.026 seconds

Signal parameter estimation through hierarchical conjugate gradient least squares applied to tensor decomposition

  • Liu, Long;Wang, Ling;Xie, Jian;Wang, Yuexian;Zhang, Zhaolin
    • ETRI Journal
    • /
    • v.42 no.6
    • /
    • pp.922-931
    • /
    • 2020
  • A hierarchical iterative algorithm for the canonical polyadic decomposition (CPD) of tensors is proposed by improving the traditional conjugate gradient least squares (CGLS) method. Methods based on algebraic operations are investigated with the objective of estimating the direction of arrival (DoA) and polarization parameters of signals impinging on an array with electromagnetic (EM) vector-sensors. The proposed algorithm adopts a hierarchical iterative strategy, which enables the algorithm to obtain a fast recovery for the highly collinear factor matrix. Moreover, considering the same accuracy threshold, the proposed algorithm can achieve faster convergence compared with the alternating least squares (ALS) algorithm wherein the highly collinear factor matrix is absent. The results reveal that the proposed algorithm can achieve better performance under the condition of fewer snapshots, compared with the ALS-based algorithm and the algorithm based on generalized eigenvalue decomposition (GEVD). Furthermore, with regard to an array with a small number of sensors, the observed advantage in estimating the DoA and polarization parameters of the signal is notable.

Modified partial least squares method implementing mixed-effect model

  • Kyunga Kim;Shin-Jae Lee;Soo-Heang Eo;HyungJun Cho;Jae Won Lee
    • Communications for Statistical Applications and Methods
    • /
    • v.30 no.1
    • /
    • pp.65-73
    • /
    • 2023
  • Contemporary biomedical data often involve an ill-posed problem owing to small sample size and large number of multi-collinear variables. Partial least squares (PLS) method could be a plausible alternative to an ill-conditioned ordinary least squares. However, in the case of a PLS model that includes a random-effect, how to deal with a random-effect or mixed effects remains a widely open question worth further investigation. In the present study, we propose a modified multivariate PLS method implementing mixed-effect model (PLSM). The advantage of PLSM is its versatility in handling serial longitudinal data or its ability for taking a randomeffect into account. We conduct simulations to investigate statistical properties of PLSM, and showcase its real clinical application to predict treatment outcome of esthetic surgical procedures of human faces. The proposed PLSM seemed to be particularly beneficial 1) when random-effect is conspicuous; 2) the number of predictors is relatively large compared to the sample size; 3) the multicollinearity is weak or moderate; and/or 4) the random error is considerable.

A PRECONDITIONER FOR THE LSQR ALGORITHM

  • Karimi, Saeed;Salkuyeh, Davod Khojasteh;Toutounian, Faezeh
    • Journal of applied mathematics & informatics
    • /
    • v.26 no.1_2
    • /
    • pp.213-222
    • /
    • 2008
  • Iterative methods are often suitable for solving least squares problems min$||Ax-b||_2$, where A $\epsilon\;\mathbb{R}^{m{\times}n}$ is large and sparse. The well known LSQR algorithm is among the iterative methods for solving these problems. A good preconditioner is often needed to speedup the LSQR convergence. In this paper we present the numerical experiments of applying a well known preconditioner for the LSQR algorithm. The preconditioner is based on the $A^T$ A-orthogonalization process which furnishes an incomplete upper-lower factorization of the inverse of the normal matrix $A^T$ A. The main advantage of this preconditioner is that we apply only one of the factors as a right preconditioner for the LSQR algorithm applied to the least squares problem min$||Ax-b||_2$. The preconditioner needs only the sparse matrix-vector product operations and significantly reduces the solution time compared to the unpreconditioned iteration. Finally, some numerical experiments on test matrices from Harwell-Boeing collection are presented to show the robustness and efficiency of this preconditioner.

  • PDF

Milling tool wear forecast based on the partial least-squares regression analysis

  • Xu, Chuangwen;Chen, Hualing
    • Structural Engineering and Mechanics
    • /
    • v.31 no.1
    • /
    • pp.57-74
    • /
    • 2009
  • Power signals resulting from spindle and feed motor, present a rich content of physical information, the appropriate analysis of which can lead to the clear identification of the nature of the tool wear. The partial least-squares regression (PLSR) method has been established as the tool wear analysis method for this purpose. Firstly, the results of the application of widely used techniques are given and their limitations of prior methods are delineated. Secondly, the application of PLSR is proposed. The singular value theory is used to noise reduction. According to grey relational degree analysis, sample variable is filtered as part sample variable and all sample variables as independent variables for modelling, and the tool wear is taken as dependent variable, thus PLSR model is built up through adapting to several experimental data of tool wear in different milling process. Finally, the prediction value of tool wear is compare with actual value, in order to test whether the model of the tool wear can adopt to new measuring data on the independent variable. In the new different cutting process, milling tool wear was predicted by the methods of PLSR and MLR (Multivariate Linear Regression) as well as BPNN (BP Neural Network) at the same time. Experimental results show that the methods can meet the needs of the engineering and PLSR is more suitable for monitoring tool wear.

EVALUATION OF PARAMETER ESTIMATION METHODS FOR NONLINEAR TIME SERIES REGRESSION MODELS

  • Kim, Tae-Soo;Ahn, Jung-Ho
    • Journal of applied mathematics & informatics
    • /
    • v.27 no.1_2
    • /
    • pp.315-326
    • /
    • 2009
  • The unknown parameters in regression models are usually estimated by using various existing methods. There are several existing methods, such as the least squares method, which is the most common one, the least absolute deviation method, the regression quantile method, and the asymmetric least squares method. For the nonlinear time series regression models, which do not satisfy the general conditions, we will compare them in two ways: 1) a theoretical comparison in the asymptotic sense and 2) an empirical comparison using Monte Carlo simulation for a small sample size.

  • PDF

Quasi-Likelihood Approach for Linear Models with Censored Data

  • Ha, Il-Do;Cho, Geon-Ho
    • Journal of the Korean Data and Information Science Society
    • /
    • v.9 no.2
    • /
    • pp.219-225
    • /
    • 1998
  • The parameters in linear models with censored normal responses are usually estimated by the iterative maximum likelihood and least square methods. However, the iterative least square method is simple but hardly has theoretical justification, and the iterative maximum likelihood estimating equations are complicatedly derived. In this paper, we justify these methods via Wedderburn (1974)'s quasi-likelihood approach. This provides an explicit justification for the iterative least square method and also directly the iterative maximum likelihood method for estimating the regression coefficients.

  • PDF

Comparison between Total Least Squares and Ordinary Least Squares for Linear Relationship of Stable Water Isotopes (완전최소자승법과 보통최소자승법을 이용한 물안정동위원소의 선형관계식 비교)

  • Lee, Jeonghoon;Choi, Hye-Bin;Lee, Won Sang;Lee, Seung-Gu
    • Economic and Environmental Geology
    • /
    • v.50 no.6
    • /
    • pp.517-523
    • /
    • 2017
  • A linear relationship between two stable water isotopes, oxygen and hydrogen, has been used to understand the water cycle as a basic tool. A slope and intercept from the linear relationship indicates what kind of physical processes occur during movement of water. Traditionally, ordinary least squares (OLS) method has been utilized for the linear relationship, but total least squares (TLS) method provides more accurate slope and intercept theoretically because isotopic compositions of both oxygen and hydrogen have uncertainties. In this work, OLS and TLS were compared with isotopic compositions of snow and snowmelt collected from the King Sejong Station, Antarctica and isotopic compositions of water vapor observed by Lee et al. (2013) in the western part of Korea. The slopes from the linear relationship of isotopic compositions of snow and snowmelt at the King Sejong Station were estimated to be 7.00 (OLS) and 7.16(TLS) and the slopes of stable water vapor isotopes were 7.75(OLS) and 7.87(TLS). There was a melting process in the snow near the King Sejong Station and the water vapor was directly transported from the ocean to the study area based on the slope calculations. There is no significant difference in two slopes to interpret the physical processes. However, it is necessary to evaluate the slope differences from the two methods for studies for example, groundwater recharge processes, using the absolute slope values.

Gas-liquid interface treatment in underwater explosion problem using moving least squares-smoothed particle hydrodynamics

  • Hashimoto, Gaku;Noguchi, Hirohisa
    • Interaction and multiscale mechanics
    • /
    • v.1 no.2
    • /
    • pp.251-278
    • /
    • 2008
  • In this study, we investigate the discontinuous-derivative treatment at the gas-liquid interface in underwater explosion (UNDEX) problems by using the Moving Least Squares-Smoothed Particle Hydrodynamics (MLS-SPH) method, which is known as one of the particle methods suitable for problems where large deformation and inhomogeneity occur in the whole domain. Because the numerical oscillation of pressure arises from derivative discontinuity in the UNDEX analysis using the standard SPH method, the MLS shape function with Discontinuous-derivative Basis Function (DBF) that is able to represent the derivative discontinuity of field function is utilized in the MLS-SPH formulation in order to suppress the nonphysical pressure oscillation. The effectiveness of the MLS-SPH with DBF is demonstrated in comparison with the standard SPH and conventional MLS-SPH though a shock tube problem and benchmark standard problems of UNDEX of a trinitrotoluene (TNT) charge.

FINITE ELEMENT APPROXIMATION OF THE DISCRETE FIRST-ORDER SYSTEM LEAST SQUARES FOR ELLIPTIC PROBLEMS

  • SHIN, Byeong-Chun
    • Communications of the Korean Mathematical Society
    • /
    • v.20 no.3
    • /
    • pp.563-578
    • /
    • 2005
  • In [Z. Cai and B. C. Shin, SIAM J. Numer. Anal. 40 (2002), 307-318], we developed the discrete first-order system least squares method for the second-order elliptic boundary value problem by directly approximating $H(div){\cap}H(curl)-type$ space based on the Helmholtz decomposition. Under general assumptions, error estimates were established in the $L^2\;and\;H^1$ norms for the vector and scalar variables, respectively. Such error estimates are optimal with respect to the required regularity of the solution. In this paper, we study solution methods for solving the system of linear equations arising from the discretization of variational formulation which possesses discrete biharmonic term and focus on numerical results including the performances of multigrid preconditioners and the finite element accuracy.

A FAST KACZMARZ-KOVARIK ALGORITHM FOR CONSISTENT LEAST-SQUARES PROBLEMS

  • Popa, Constantin
    • Journal of applied mathematics & informatics
    • /
    • v.8 no.1
    • /
    • pp.9-26
    • /
    • 2001
  • In some previous papers the author extended two algorithms proposed by Z. Kovarik for approximate orthogonalization of a finite set of linearly independent vectors from a Hibert space, to the case when the vectors are rows (not necessary linearly independent) of an arbitrary rectangular matrix. In this paper we describe combinations between these two methods and the classical Kaczmarz’s iteration. We prove that, in the case of a consistent least-squares problem, the new algorithms so obtained converge ti any of its solutions (depending on the initial approximation). The numerical experiments described in the last section of the paper on a problem obtained after the discretization of a first kind integral equation ilustrate the fast convergence of the new algorithms. AMS Mathematics Subject Classification : 65F10, 65F20.