Smoothness Priors Modeling

Time Series Analysis and Control Examples

Smoothness Priors Modeling

Consider the time series y_t:

$y_t = f(t) + \epsilon_t$

where f(t) is an unknown smooth function and $\epsilon_t$ is an iid random variable with zero mean and positive variance $\sigma^2$ . Whittaker (1923) provides the solution, which balances a tradeoff between closeness to the data and the kth order difference equation. For a fixed value of $\lambda$ and k, the solution $\hat{f}$ satisfies

$\min_f \sum_{t=1}^T \{ [y_t - f(t)]^2 + \lambda^2 [\nabla^k f(t)]^2 \}$

where $\nabla^k$ denotes the kth order difference operator. The value of $\lambda$ can be viewed as the smoothness tradeoff measure. Akaike (1980a) proposed the Bayesian posterior PDF to solve this problem.

$\ell(f) = \exp \{ -\frac{1}{2 \sigma^2} \sum_{t=1}^T [y_t - f(t)]^2 \} \exp \{ -\frac{\lambda^2}{2 \sigma^2} \sum_{t=1}^T [\nabla^k f(t)]^2 \}$

Therefore, the solution can be obtained when the function l(f) is maximized.

Assume that time series is decomposed as follows:

$y_t = T_t + S_t + \epsilon_t$

where T_t denotes the trend component and S_t is the seasonal component. The trend component follows the kth order stochastically perturbed difference equation.

$\nabla^k T_t = w_{1t}, \hspace*{0.25in} w_{1t} \sim N(0,\tau_1^2)$

For example, the polynomial trend component for k=2 is written as

T_t = 2T_t-1 - T_t-2 + w_1t

To accommodate regular seasonal effects, the stochastic seasonal relationship is used.

$\sum_{i=0}^{L-1} S_{t-i} = w_{2t}, \hspace*{0.25in} w_{2t} \sim N(0,\tau_2^2)$

where L is the number of seasons within a period. In the context of Whittaker and Akaike, the smoothness priors problem can be solved by the maximization of

$\ell(f) & = & \exp [ -\frac{1}{2 \sigma^2} \sum_{t=1}^T (y_t - T_t - S_t)^2 ... ...-\frac{\tau_2^2}{2 \sigma^2} \sum_{t=1}^T ( \sum_{i=0}^{L-1} S_{t-i} )^2 ]$

The values of hyperparameters, $\tau_1^2$ and $\tau_2^2$ ,refer to a measure of uncertainty of prior information. For example, the large value of $\tau_1^2$ implies a relatively smooth trend component. The ratio $\frac{\tau_i^2}{\sigma^2}\; (i=1,2)$ can be considered as a signal-to-noise ratio.

Kitagawa and Gersch (1984) use the Kalman filter recursive computation for the likelihood of the tradeoff parameters. The hyperparameters are estimated by combining the grid search and optimization method. The state space model and Kalman filter recursive computation are discussed in the section, "State Space and Kalman Filter Method".

Chapter Contents
Previous
Next
Top