An assessment of full cross-validation

by Mark C. Greenwood and Glenna Gordon.

Abstract: Full cross-validation was promoted as an alternative to regular cross-validation for nonlinear regression model selection in Bunke et al. (1998, 1999). In Droge (1995), simulations were performed to explore its performance for model selection in a polynomial regression context, finding mixed results at best. The poor performance of the method was not highlighted in later publications related to the method. In this work, we explore its performance for nonlinear regression models, which has not been evaluated previously. The method is attractive for use in situations where cross-validation methods are desired but estimation algorithms are not easily modified for missing observations or estimation can easily diverge when design points are removed, such as nonlinear regression. A simulation study is used to reinforce the poor performance of FCV for model selection in linear regression and to demonstrate that its problems extend into nonlinear regression models as well. For moderate sample sizes in linear regression, the problems with FCV seem to diminish but the protection of a larger sample size seems to disappear for the nonlinear regression models explored. This suggests caution in using FCV for model selection in general.

Key Words: Model Selection, Cross Validation, Full Cross Validation, Linear Regression, Nonlinear Regression

Authors:
Mark C. Greenwood, greenwood@math.montana.edu
Glenna Gordon, glenna.gordon@iem.com

Editor: R.G. Graf, rgraf@sunstroke.sdsu.edu

READING THE ARTICLE: You can read the article in portable document (.pdf) format (165997 bytes.)

NOTE: The content of this article is the intellectual property of the authors, who retains all rights to future publication.

This page has been accessed 1783 times since AUGUST 14, 2010.


Return to the InterStat Home Page.