Error Covariance Structure Specification

The MODEL Procedure

Error Covariance Structure Specification

One of the key assumptions of regression is that the variance of the errors is constant across observations. Correcting for heteroscedasticity improves the efficiency of the estimates.

Consider the following general form for models:

$q(y_t, x_t, \theta) &=& {\varepsilon_t} \{\varepsilon_t } &=& H_t * {\epsilon_t}... ... & & \ddots & \ 0 & 0 & ... & \sqrt{h_{t,g}} ] \{h_t} &=& g( y_t, x_t, \phi )$

where ${\epsilon_t} \sim N(0,\Sigma)$ .

For models which are homoscedastic

h_t = 1.

(14.1)

If you had a model which was heteroscedastic with known form you can improve the efficiency of the estimates by performing a weighted regression. The weight variable, using this notation, would be $1/\sqrt{h_t}$ .

If the errors for a model are heteroscedastic and the functional form of the variance is known, the model for the variance can now be estimated along with the regression function.

To specify a functional form for the variance, assign the function to an H.var variable where var is the equation variable. For example, if you wanted to estimate the scale parameter for the variance of a simple regression model

y = a *x + b

(14.2)

you can specify

   proc model data=s;
      y = a * x + b;
      h.y = sigma**2;
   fit y;

Consider the same model with the following functional form for the variance:

$h_t = \sigma^2 * x ^{2*\alpha}$ (14.3)

this would be written as:

   proc model data=s;
      y = a * x + b;
      h.y = sigma**2 * x**(2*alpha);
   fit y;

There are three ways to model the variance in the MODEL procedure; Feasable generalized least squares; Generalized method of moments; and Full information maximum likelihood.

Feasable GLS

A simple approach to estimating a variance function is to estimate the mean parameters ${\theta}$ using some auxilary method, such as OLS, and then use the residuals of that estimation to estimate the parameters $\phi$ of the variance function. This scheme is called feasable GLS. It is posible to use the residuals from an auxilary method for the purpose of estimating $\phi$ because in many cases the residuals consistently estimate the error terms.

This scheme can be done by hand by performing OLS estimation of $q(y_t, x_t, \theta)$ , the mean function, then by regressing the residuals squared on h_t, and finally by re-estimating the the mean function using a weight of $1/\sqrt{h_t}$ .

For all estimation methods except GMM and FIML, using the H.var syntax specifies that feasable GLS will be used in the estimation. For feasable GLS the mean function is estimated by the usual method. The variance function is then estimated using pseudolikelihood (PL) function of the generated residuals. The objective function for the PL estimation is

$p_n( \sigma, \theta ) = \sum_{i=1}^n ( \frac{ (y_i-f(x_i,\hat{\beta}))^2}{\sigma^2 h( z_i, \theta) } + \log[ \sigma^2 h( z_i, \theta) ] )$ (14.4)

Once the variance function has been estimated the mean function is re-estimated using the variance function as weights. If an S-iterated method is selected, this process is repeated until convergence (iterated feasable GLS).

Note, feasable GLS will not yield consistent estimates when one of the following is true:

The variance is unbounded.
There is too much serial dependence in the errors (the dependence does not fade with time).
A combination of serial dependence and lag dependent variables.

The first two cases are unusual but the third is much more common. Whether iterated feasable GLS avoids consistency problems with the last case is an unanswered research question. For more information see (Davidson and MacKinnon 1993) pages 298-301 or (Gallant 1987) pages 124-125 and (Amemiya 1985) pages 202-203.

One limitation is that parameters can not be shared between the mean equation and the variance equation. This implies that certian GARCH models, cross equation restrictions of parameters, or testing of combinations of parameters in the mean and variance component are not allowed.

Generalized Method of Moments

In GMM, normally the first moment of the mean function is used in the objective function.

$q(y_t, x_t, \theta) &=& \epsilon_t \E (\epsilon_t) &=& 0$

To add the second moment conditions to the estimation, add the equation

$E(\varepsilon_t * \varepsilon_t - h_t ) = 0$

to the model. For example if you wanted to estimate ${{\sigma}}$ for linear example above, you can write

   proc model data=s;
      y = a * x + b;
      eq.two = resid.y**2 - sigma**2;
   fit y two/ gmm;
   instruments x;
   run;

This is a popular way to estimate a continuous-time interest rate processes (see (Chan, et al 1992)). The H.var syntax will automatically generate this system of equations.

To further take advantage of the information obtained about the variance, the moment equations can be modified to

$E( \varepsilon_t / \sqrt{ h_t}) &=& 0 \E( \varepsilon_t * \varepsilon_t - h_t ) &=& 0$

For the above example, this can be written as

   proc model data=s;
      y = a * x + b;
      eq.two = resid.y**2 - sigma**2;
      resid.y = resid.y / sigma;
   fit y two/ gmm;
   instruments x;
   run;

Note, if the error model is misspecified in this form of the GMM model the parameter estimates may be inconsistent.

Full Information Maximum Likelihood

For FIML estimation of variance functions, the concentrated likelihood below is used as the objective function. That is, the mean function will be coupled with the variance function and the system will be solved simultaneously.

$l_n(\phi) &=& \frac{ng}2(1+\ln(2\pi)) - \sum_{t=1}^n \ln ( | \frac{\partial q(y_... ...\sum_{t=1}^n \sum_{i=1}^g ( \ln( h_{t,i}) + q_i(y_t, x_t, \theta)^2 / h_{t,i} )$

where g is the number of equations in the system.

The HESSIAN=GLS option is not available for FIML estimation involving variance functions. The matrix used when HESSIAN=CROSS is specified is a cross products matrix which has been enhanced by the dual quasi-newton approximation.

Examples

You can specify a GARCH(1,1) model as follows:

   proc model data=modloc.usd_jpy;
   
                /* Mean model --------*/
      jpyret = intercept ;
   
                /* Variance model ----------------*/
      h.jpyret = arch0 + arch1 * zlag( resid.jpyret * resid.jpyret ) +
                 garch1 * zlag(h.jpyret) ;
   
      bounds arch0 arch1 garch1 >= 0;
   
   fit jpyret/method=marquardt fiml;
   run;

Note that the BOUNDS statement was used to ensure that the parameters were positive, a requirement for GARCH models.

EGARCH models are used because there is no restrictions on the parameters. You can specify a EGARCH(1,1) model as follows:

   proc model data=sasuser.usd_dem ;
   
             /* Mean model ----------*/
     demret = intercept ;
   
                /* Variance model ----------------*/
     if ( _OBS_ =1 )  then
       h.demret = exp( earch0/ (1. - egarch1)  );
     else
       h.demret = exp( earch0 + earch1 * zlag( g)
                            + egarch1 * log(zlag(h.demret)));
     g = theta * nresid.demret + abs( nresid.demret ) - sqrt(2/3.1415);
   
                        /* Fit and save the model */
   fit demret/method=marquardt fiml  maxiter=100
   run;

Chapter Contents
Previous
Next
Top