Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

An EM Algorithm for a Doubly Smoothed MLE in Normal Mixture Models

  • Received : 20110900
  • Accepted : 20111100
  • Published : 2012.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

It is well known that the maximum likelihood estimator(MLE) in normal mixture models with unequal variances does not fall in the interior of the parameter space. Recently, a doubly smoothed maximum likelihood estimator(DS-MLE) (Seo and Lindsay, 2010) was proposed as a general alternative to the ordinary maximum likelihood estimator. Although this method gives a natural modification to the ordinary MLE, its computation is cumbersome due to intractable integrations. In this paper, we derive an EM algorithm for the DS-MLE under normal mixture models and propose a fast computational tool using a local quadratic approximation. The accuracy and speed of the proposed method is then presented via some numerical studies.

Keywords

References

  1. Chen, J., Tan, X. and Zhang, R. (2008). Consistency of penalized mle for normal mixtures in mean and variance, Statistica Sinica, 18, 443-465.
  2. Ciuperca, G. A., Ridolfi, A. and Idier, J. (2003). Penalized maximum likelihood estimator for normal mixtures, Scandinavian Journal of Statistics, 30, 45-59. https://doi.org/10.1111/1467-9469.00317
  3. Crawford, S. K., Degroot, M. H., Kadane, J. B. and Small, M. J. (1992). Modeling lake chemistry distributions: Approximate Bayesian methods for estimating a finite mixture model, Technometrics, 34, 441-453. https://doi.org/10.2307/1268943
  4. Fraley, C. and Raftery, A. E. (2007). Bayesian regularization for normal mixture estimation and model-based clustering, Journal of Classification, 24, 155-181. https://doi.org/10.1007/s00357-007-0004-5
  5. Hathaway, R. J. (1985). A constrained formulation of maximum likelihood estimation for normal mixture distributions, Annals of Statistics, 13, 795-800. https://doi.org/10.1214/aos/1176349557
  6. Hathaway, R. J. (1986). A constrained EM-algorithm for univariate normal mixtures, Computational Statistics & Data Analysis, 23, 211-230.
  7. Ingrassia, S. and Rocci, R. (2007). Constrained monotone EM algorithms for finite mixture of multi- variate Gaussians, Computational Statistics & Data Analysis, 51, 5339-5351. https://doi.org/10.1016/j.csda.2006.10.011
  8. Kiefer, J. andWolfowitz, J. (1956). Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters, Annals of Statistics, 27, 886-906.
  9. Lindsay, B. G., Markatou, M., Ray, S., Yang, K. and Chen, S. (2008). Quadratic distances on probabilities: A unified foundation, Annals of Statistics, 36, 983-1006. https://doi.org/10.1214/009053607000000956
  10. Ray, S. and Lindsay, B. G. (2008). Model selection in high-dimensions: A quadratic-risk based approach, Journal of the Royal Statistical Society, Series B, 70, 95-118.
  11. Seo, B. and Lindsay, B. G. (2010). A computational strategy for doubly smoothed MLE exemplified in the normal mixture model, Computational Statistics and Data Analysis, 54, 1930-1941. https://doi.org/10.1016/j.csda.2010.02.026
  12. Seo, B. and Lindsay, B. G. (2011). A universally consistent modification of maximum likelihood, Submitted.
  13. Tanaka, K. and Takemura, A. (2006). Strong consistency of the maximum likelihood estimator for finite mixtures of location-scale distributions when the scale parameters are exponentially small, Bernoulli, 12, 1003-1017. https://doi.org/10.3150/bj/1165269148