Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

A Comparative Study on the Performance of Bayesian Partially Linear Models

  • Received : 2012.08.20
  • Accepted : 2012.11.13
  • Published : 2012.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we consider Bayesian approaches to partially linear models, in which a regression function is represented by a semiparametric additive form of a parametric linear regression function and a nonparametric regression function. We make a comparative study on the performance of widely used Bayesian partially linear models in terms of empirical analysis. Specifically, we deal with three Bayesian methods to estimate the nonparametric regression function, one method using Fourier series representation, the other method based on Gaussian process regression approach, and the third method based on the smoothness of the function and differencing. We compare the numerical performance of three methods by the root mean squared error(RMSE). For empirical analysis, we consider synthetic data with simulation studies and real data application by fitting each of them with three Bayesian methods and comparing the RMSEs.

Keywords

References

  1. Aerts, M., Claeskens, G. and Hart, J. D. (2004). Bayesian-motivated tests of function fit and their asymptotic frequentist properties, The Annals of Statistics, 32, 2580-2615. https://doi.org/10.1214/009053604000000805
  2. Brooks, S. P. and Gelman, A. (1998). General methods for monitoring convergence of iterative simulations, Journal of Computational and Graphical Statistics, 7, 434-455.
  3. Choi, T., Lee, J. and Roy, A. (2009). A note on the Bayes factor in a semiparametric regression model, Journal of Multivariate Analysis, 100, 1316-1327. https://doi.org/10.1016/j.jmva.2008.12.006
  4. Choi, T., Shi, J. Q. andWang, B. (2011). A Gaussian process regression approach to a single-index model, Journal of Nonparametric Statistics, 23, 21-36. https://doi.org/10.1080/10485251003768019
  5. Choi, T. and Woo, Y. (2012). A partially linear model using a Gaussian process prior, submitted.
  6. Damien, P.,Wakefield, J. andWalker, S. (1999). Gibbs sampling for Bayesian non-conjugate and hierarchical models by using auxiliary variables, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61, 331-344. https://doi.org/10.1111/1467-9868.00179
  7. Engle, R. F., Granger, C. W. J., Rice, J. and Weiss, A. (1986). Semiparametric estimates of the relation between weather and electricity sales, Journal of the American Statistical Association, 81, 310-320. https://doi.org/10.1080/01621459.1986.10478274
  8. H¨ardle, W., Liang, H. and Gao, J. (2000). Partially linear Models, Physica-Verlag, Heidelberg.
  9. Hayfield, T. and Racine, J. S. (2008). Nonparametric econometrics: The np package, Journal of Statistical Software, 27, 1-32.
  10. Kennedy, M. C. and O'Hagan, A. (2001). Bayesian calibration of computer models, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63, 425-464. https://doi.org/10.1111/1467-9868.00294
  11. Kneib, T., Konrath, S. and Fahrmeir, L. (2011). High dimensional structured additive regression models: Bayesian regularization, smoothing and predictive performance, Journal of the Royal Statistical Society: Series C (Applied Statistics), 60, 51-70. https://doi.org/10.1111/j.1467-9876.2010.00723.x
  12. Koop, G. and Poirier, D. J. (2004). Bayesian variants of some classical semiparametric regression techniques, Journal of Econometrics, 123, 259-282. https://doi.org/10.1016/j.jeconom.2003.12.008
  13. Lenk, P. J. (1999). Bayesian inference for semiparametric regression using a Fourier representation, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61, 863-879. https://doi.org/10.1111/1467-9868.00207
  14. Li, Q. and Racine, J. S. (2007). Nonparametric Econometrics, Theory and Practice, Princeton University Press, Princeton, New Jersey.
  15. Lindley, D. V. and Smith, A. F. M. (1972). Bayes estimates for the linear model, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 34, 1-41.
  16. Na, J. and Kim, J. (2002). Bayesian model selection and diagnostics for nonlinear regression model, Korean Journal of Applied Statistics, 15, 139-151. https://doi.org/10.5351/KJAS.2002.15.1.139
  17. Oakley, J. E. and O'Hagan, A. (2004). Probabilistic sensitivity analysis of complex models: A Bayesian approach, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 66, 751-769. https://doi.org/10.1111/j.1467-9868.2004.05304.x
  18. O'Hagan, A. (1978). Curve fitting and optimal design for prediction, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 40, 1-42.
  19. Rasmussen, C. E. and Williams, C. K. I. (2006). Gaussian Processes for Machine Learning, MIT Press, Cambridge, MA.
  20. Ruppert, D., Wand, M. P. and Caroll, R. J. (2009). Semiparametric regression during 2003-2007, Electronic Journal of Statistics, 3, 1193-1256. https://doi.org/10.1214/09-EJS525
  21. Shi, J. Q. and Choi, T. (2011). Gaussian Process Regression Analysis for Functional Data, Chapman & Hall/CRC Press, New York.
  22. Shi, J. Q., Murray-Smith, R. and Titterington, D. M. (2007). Gaussian process function regression modeling for batch data, Biometrics, 63, 714-723. https://doi.org/10.1111/j.1541-0420.2007.00758.x
  23. Shi, J. Q. and Wang, B. (2008). Curve prediction and clustering with mixtures of Gaussian process functional and regression models, Statistics and Computing, 18, 267-283. https://doi.org/10.1007/s11222-008-9055-1
  24. Wooldridge, J. M. (2003). Introductory Econometrics, A Modern Approach, MIT Press, Cambridge, MA.
  25. Yatchew, A. (1998). Nonparametric regression technique in Economics, Journal of Economic Literature, 36, 669-721.
  26. Yi, G., Shi, J. Q. and Choi, T. (2011). Penalized Gaussian process regression and classification for highdimensional nonlinear data, Biometrics, 67, 1285-1294. https://doi.org/10.1111/j.1541-0420.2011.01576.x
  27. Yu, Y. and Ruppert, D. (2002). Penalized spline estimation for partially linear single-index models, Journal of the American Statistical Association, 97, 1042-1054. https://doi.org/10.1198/016214502388618861