UFORECAST: BOOTSTRAP

[ac_1]

Hi Tom,

https://estima.com/ratshelp/index.html? ... ction.html

says

BOOTSTRAP/[NOBOOTSTRAP]
BOOTSTRAP draws shocks over the forecast period randomly with replacement from the residuals associated with the equation or regression being forecast.Because this is a simple shuffling of the residuals, it would not be completely appropriate for a model with moving average terms if you’re bootstrapping an entire sample.

Does this mean bootstrapping cannot be applied to MA models or ARIMA models which include MA terms?

thanks,
Amarjit

[TomDoan]

See https://www.estima.com/forum/viewtopic. ... 55&p=17220

[ac_1]

Hi Tom,

My aim is to generate automated model selection BOOTSTRAP multi-step & one-step ahead OOS forecasts with PI's for ANY non-seasonal and seasonal ARIMA model, according to:
- @BJTRANS - choose a preliminary transformation
- @BJDIFF - automated choice for the DIFFS, SDIFFS and CONSTANT
- @GMAUTOFIT - automatic fit of a multiplicative seasonal ARMA model
- Estimate model and generate forecasts with PI's

I have followed the income.rpf example https://estima.com/ratshelp/index.html?bootarmarpf.html and can generate ndraws of DYBOOT as a vect[series] fc (say), via a double loop: draw and time, before the Kalman gain, thereafter calculate mean & se in the usual way until %regend().

Steps and Questions:
(S1) @BJTRANS suggests a sqrt transformation: -258.44 vs -261.64 (None), -264.229 (Log). Why choose log?
(S2) Choosing log, @BJDIFF says no Reg Diff, No Seasonal Diff, Yes to include a constant. Why estimate the ARMA model in differenced form and why no constant?
(S3) Then compute the residuals and the predictive variances:
- convert to SS representation, and
- use DLM i.e. Kalman filter, twice.
(S4) Thereafter, generate a cloud of simulations fc (say)
(S5) Calculate mean & se of BOOTSTRAP forecasts, at EACH time step, but all IS only, and plot forecasts. How do I generate multi-step & one-step ahead OOS forecasts with PI's?

Amarjit

[TomDoan]

Hi Tom,

My aim is to generate automated model selection BOOTSTRAP multi-step & one-step ahead OOS forecasts with PI's for ANY non-seasonal and seasonal ARIMA model, according to:
- @BJTRANS - choose a preliminary transformation
- @BJDIFF - automated choice for the DIFFS, SDIFFS and CONSTANT
- @GMAUTOFIT - automatic fit of a multiplicative seasonal ARMA model
- Estimate model and generate forecasts with PI's

I have followed the income.rpf example https://estima.com/ratshelp/index.html?bootarmarpf.html and can generate ndraws of DYBOOT as a vect[series] fc (say), via a double loop: draw and time, before the Kalman gain, thereafter calculate mean & se in the usual way until %regend().

Steps and Questions:
(S1) @BJTRANS suggests a sqrt transformation: -258.44 vs -261.64 (None), -264.229 (Log). Why choose log?

That's pretty much automatic for income. Does sqrt seem to fit better over this particular set of data? Yes. Would the same be true for the same series if it were extended another 10 years? Probably not. (This is a particularly messy stretch of data with multiple recessions and a change in the growth rate halfway through the sample).

(S2) Choosing log, @BJDIFF says no Reg Diff, No Seasonal Diff, Yes to include a constant. Why estimate the ARMA model in differenced form and why no constant?

If you read the comments, the "constant" is handled by de-meaning the data.

(S3) Then compute the residuals and the predictive variances:
- convert to SS representation, and
- use DLM i.e. Kalman filter, twice.
(S4) Thereafter, generate a cloud of simulations fc (say)
(S5) Calculate mean & se of BOOTSTRAP forecasts, at EACH time step, but all IS only, and plot forecasts. How do I generate multi-step & one-step ahead OOS forecasts with PI's?

Amarjit

This does one bootstrap replication over the full data range. It's designed for analyzing things like the parameter estimates. The difficult part of that is the front end of the data series where there is almost no information. Forecasting is a completely different operation since you are starting well into the data set and treating observed data as given, so this is massive overkill for that. All you need from this is the STDRESIDS series. Then you can replicate over FORECAST on the ARMA model with the PATHS option, where the PATHS series are just reflated (multiplied by sqrt(%seesq)) draws from STDRESIDS. Look at other example of out-of-sample bootstrapping, where you draw residuals/standardized residuals from the sample range and use them over a different range.

[ac_1]

(S2) Choosing log, @BJDIFF says no Reg Diff, No Seasonal Diff, Yes to include a constant. Why estimate the ARMA model in differenced form and why no constant?

If you read the comments, the "constant" is handled by de-meaning the data.

I see the extracted mean, but why estimate in differenced form?

The bootstrapped forecasts (and PI's) should be for the original series, unlogged and in levels. Based on the example in https://www.estima.com/ratshelp/index.h ... ction.html, if I try

Code: Select all

clear dysimul
boot select / bstart bend
set path = stdresids(select)*sqrt(%seesq)

prin / stdresids path

forecast(paths,from=1985:2+1,steps=10) 1
# dyeq dysimul
# path

there is nothing in dysimul?

[TomDoan]

(S2) Choosing log, @BJDIFF says no Reg Diff, No Seasonal Diff, Yes to include a constant. Why estimate the ARMA model in differenced form and why no constant?

If you read the comments, the "constant" is handled by de-meaning the data.

I see the extracted mean, but why estimate in differenced form?

The bootstrapped forecasts (and PI's) should be for the original series, unlogged and in levels. Based on the example in https://www.estima.com/ratshelp/index.h ... ction.html, if I try

Code: Select all

clear dysimul
boot select / bstart bend
set path = stdresids(select)*sqrt(%seesq)

prin / stdresids path

forecast(paths,from=1985:2+1,steps=10) 1
# dyeq dysimul
# path

there is nothing in dysimul?

It's estimated in differences because it's bootstrapping for an *ARMA* model. The analysis doesn't work for a model with unit roots. But, of course, you can always integrate up the differenced data.

In your bootstrapping, your source for the entries is the estimated sample; the target is the forecast range. You aren't being careful about that.

[ac_1]

From reading the BOOT instruction

Code: Select all

compute span=10

set path bend+1 bend+span = path(bend)

boot select bend+1 bend+span bstart bend
set path bend+1 bend+span = stdresids(select)*sqrt(%seesq)

prin bstart bend+span stdresids path

forecast(paths,from=bend+1,to=bend+span) 1
# dyeq dysimul
# path

graph 2
# dy bstart bend
# dysimul

produces reasonable point forecasts (differenced and logged).

However,

My aim is to generate automated model selection BOOTSTRAP multi-step & one-step ahead OOS forecasts with PI's for ANY non-seasonal and seasonal ARIMA model, according to:
- @BJTRANS - choose a preliminary transformation
- @BJDIFF - automated choice for the DIFFS, SDIFFS and CONSTANT
- @GMAUTOFIT - automatic fit of a multiplicative seasonal ARMA model
- Estimate model and generate forecasts with PI's

[TomDoan]

@BJTRANS isn't designed to provide an automated answer. Do you really have data for which that's an issue?

The AUTOBOX.RPF example uses @BJDIFF and @GMAUTOFIT in an automated fashion.

What is it that you think bootstrapping this will accomplish? For point forecasts, all you are doing is injecting sampling error---with enough draws, the forecasts will converge to what you get with just a straight FORECAST instruction. Even for CI's, there is unlikely to be that much difference. (You can't repeal the Law of Large Numbers).

[ac_1]

@BJTRANS isn't designed to provide an automated answer. Do you really have data for which that's an issue?

For macro and financial data I have used NONE and LOG, rarely SQRT.

What is it that you think bootstrapping this will accomplish? For point forecasts, all you are doing is injecting sampling error---with enough draws, the forecasts will converge to what you get with just a straight FORECAST instruction. Even for CI's, there is unlikely to be that much difference. (You can't repeal the Law of Large Numbers).

For residuals with non-normal distributions I have used bootstrapping. Are you saying FORECAST is a better bet than UFORECAST for generating bootstrap forecasts for ANY ARIMA model?

[TomDoan]

What is it that you think bootstrapping this will accomplish? For point forecasts, all you are doing is injecting sampling error---with enough draws, the forecasts will converge to what you get with just a straight FORECAST instruction. Even for CI's, there is unlikely to be that much difference. (You can't repeal the Law of Large Numbers).

For residuals with non-normal distributions I have used bootstrapping. Are you saying FORECAST is a better bet than UFORECAST for generating bootstrap forecasts for ANY ARIMA model?

An ARIMA model has linear forecasts which can be computed analytically. Using bootstrapping or simulation methods for estimating that is just injecting sampling error into it. If you do 1000000 simulations, you'll probably hit the FORECAST values to four or five significant digits.

[ac_1]

An ARIMA model has linear forecasts which can be computed analytically. Using bootstrapping or simulation methods for estimating that is just injecting sampling error into it. If you do 1000000 simulations, you'll probably hit the FORECAST values to four or five significant digits.

Yes, but the prediction intervals will be different if the the residuals are non-normal!

[ac_1]

For the example above to compute forecast in levels, I would add-back-up i.e. undifference, and then exp, in that order. Correct?

I think what you are saying is if specified correctly there is no need to bootstrap and calculate analytical PI's. But recently there have been very large moves in macro & financial series, and to generate PI's from models with non-Gaussian residuals, and not use dummies in the specification, I have bootstrapped. If the above example is a massive 'overkill', would UFORECAST with BOOTSTRAP, or the PATHS method with FORECAST, for ANY ARIMA model still be appropriate?

Assuming residuals with flat autocorrelations, how else to handle non-Gaussian residuals w.r.t. forecasting PI's?

[TomDoan]

I think you'll find that there's not that big a difference. (The estimated variance in the estimation will pick up the fact that there are some big values). When you are calculating moments by simulation methods the LLN will win out in the end. Bootstrapping/simulation has more of a point when you are interested in tail events (for instance VaR calculations). Note however, that your plan is to do the following:

1. Automatically pick the transformation (choice will affect the forecasts/uncertainty)
2. Automatically pick the differencing (choice will affect the forecasts/uncertainty, particularly at longer horizons)
3. Automatically pick the model form (choice will affect the forecasts/uncertainty)
4. Estimate the model (sampling error will affect the forecasts/uncertainty)
5. Generate forecasts.

The only part that bootstrapping covers is #5. I'm guessing that among all of 1-5, the differences in forecast uncertainty due to fatter tailed residuals is probably the least important.