Chapter Contents

Previous

Next

Using Hold-out Samples

One important application of model fitting where the period of fit is different from the period of evaluation is the use of hold-out samples. With this technique of model evaluation, the period of fit ends at a time point before the end of the data series, and the remainder of the data are held out as a nonoverlapping period of evaluation. With respect to the period of fit, the hold-out sample is a period in the future, used to compare the forecasting accuracy of models fit to past data.

For this exercise, use a hold-out sample of 12 months. Bring up the Time Ranges Specification window again by selecting the Set Ranges button. Set Hold-out Sample to 12 using the combo box, as shown in Display 26.16. You can also type in a value. To specify a hold-out sample period in different units, you can use the Periods combo box. In this case, it will allow you to select years as the unit, instead of periods.

Notice that setting the holdout sample to 12 automatically sets the fit range to JUN1978--JUL1981 and the evaluation range to AUG1981--JUL1982. If you had set the period of fit and period of evaluation to these ranges, the hold-out sample would have been automatically set to 12 periods.

Select the OK button to return to the Develop Models window. Now refit the models again. Select Tools and Compare Models to compare the models now that they have been fit to the period June, 1978 through July, 1981 and evaluated for the hold-out sample period August, 1981 through July, 1982. As shown in Display 26.17, the ARIMA (0,1,0)(0,1,0)s model now seems to provide a better fit to the data than does the Airline model. It should be noted that the results can be quite different if you choose a different size hold-out sample.

Chapter Contents

Previous

Next

Top

Copyright © 1999 by SAS Institute Inc., Cary, NC, USA. All rights reserved.