Out of sample forecasting VAR

[paretto]

Dear all,
I am trying to replicate Meese and Rogoff (1983) "Empirical exchange rate models of the seventies Do they fit out of sample ?"
Journal of international Economics 14

Therefore I would like to perform an out of sample forecasting analysis to compare the forecast generated from structural models with those generated from an random walk with drift.
In the basis Theil instruction it is just possible to use a nochange random walk. But how to compare with a random walk with drift?
Therefore I decided to estimate a random walk VAR model with a constant term and compare later the root mean square errors


My code can be found below
Is this the proper way to generate forecasts for a random walk with drift?
Meese and Rogoff use future realized values of the macroeconomic fundamentals to generate forecasts for the exchange rate.
How can I specify that for a multiperiod forecast the true future realizations are used for forecasting?
Is it possible to determine that just for the economic variables the future realizations are used, while for the exchange rate dynamic predictions are used?

Thanks in advance!!!
Wolfgang

CODE:

* forecasting
do iend = ibegin,iend
system(model=varmodel)
* le is the logarithmic exchange rate
variables le sharediff idiff infdiff ydiff cadiff
lags 1 to 4
det constant
end(system)
estimate(outsigma=mysigma,coeffs=mycoeffs,residual=varresid, noprint) iend-40 iend

system(model=rawalk)
variables le
lags 1 to 1
det constant
end(system)
estimate(outsigma=mysigmarw,coeffs=mycoeffsrw,residual=varresidrw, noprint) iend-40 iend

forecast(model=rawalk,results=result) * nstep iend+1 iend+nstep
forecast(model=varmodel,results=result) * nstep iend+1 iend+nstep
end do

*rmse for structural model
theil(setup, model=varmodel, steps =nstep, to=iend)
do time=1984:01,2008:04
theil time
end
theil(dump)
* rmse for random walk
theil(setup, model=rawalk, steps =nstep, to=iend)
do time=1984:01,2008:04
theil time
end
theil(dump)

[TomDoan]

That won't give you a drifting random walk, since the AR coefficient is estimated, not fixed. (It might be quite close to 1, but won't be exactly one).

For a single series, the simplest way to estimate a drifting random walk is with BOXJENK with the DIFFS=1 and CONSTANT options. That can then be forecast with UFORECAST.

Code: Select all

boxjenk(diffs=1,constant,define=drifteq) le
uforecast(equation=drifteq,from=iend+1,steps=nstep) driftforecast

If you have multiple series, the simplest way to handle it is similar to yours, but with the LAGS statement out and a blank ECT in.

Code: Select all

system(model=rawalk)
variables le 
det constant
ect
end(system)

ECT is for error correction models, but a blank ECT is just estimating the model in first differences.

To forecast the VAR treating all other variables as "exogenous" for forecasting purposes, all you need to do is forecast the exchange rate equation in isolation. You can either estimate the one equation with LINREG, or you can estimate the VAR and use UFORECAST on an extracted equation:

Code: Select all

system(model=varmodel)
variables le sharediff idiff infdiff ydiff cadiff
lags 1 to 4
det constant
end(system)
estimate(outsigma=mysigma,coeffs=mycoeffs,residual=varresid, noprint) iend-40 iend
*
* Forecasts with all variables extrapolated dynamically
*
forecast(model=varmodel,results=result,steps=nstep,from=iend+1)
uforecast(equation=%modeleqn(varmodel,1),steps=nstep,from=iend+1) ufores

(BTW, 41 observations is a bit low for estimating that size VAR).

The simplest way to compute the forecast error statistics is with UFOREERRORS which is described on page 308 of the User's Guide.

[apollon]

Related to this topic, I have a VAR system which has monthly data say from 1997 to 2000 and I want out of sample one period forecasts between 2000 and 2005 based on a rolling regression with a moving window of 12 months.

My below (I'm sure badly written) code produces in sample forecasts fine but NA for out of sample. As a bonus, I am getting this message at the end: ## SR10. Missing Values And/Or SMPL Options Leave No Usable Data Points. I did look at the user's guide, especially in the "forecasting with rolling regressions" section but since I can't figure out what's wrong I would appreciate any help.

do regend=1998:02,2003:05
estimate(noprint) regend-12 regend
forecast(model=varmodel,results=forecasts,from=regend+1,to=regend+1,print)
end do

[TomDoan]

Related to this topic, I have a VAR system which has monthly data say from 1997 to 2000 and I want out of sample one period forecasts between 2000 and 2005 based on a rolling regression with a moving window of 12 months.

My below (I'm sure badly written) code produces in sample forecasts fine but NA for out of sample. As a bonus, I am getting this message at the end: ## SR10. Missing Values And/Or SMPL Options Leave No Usable Data Points. I did look at the user's guide, especially in the "forecasting with rolling regressions" section but since I can't figure out what's wrong I would appreciate any help.

do regend=1998:02,2003:05
estimate(noprint) regend-12 regend
forecast(model=varmodel,results=forecasts,from=regend+1,to=regend+1,print)
end do

This:

estimate(noprint) regend-12 regend

is trying to estimate your VAR with just 13 data points. I suspect that's not what you want.

[apollon]

Hi Tom

I kept 13 points as a proof of concept that the program can run, once I manage to get it working I'll change this to have more points.

I think the answer to my confusion is the way I interpreted rolling regressions. I thought I could use rolling regressions in RATS with forecasted values but there is no need to do so since the forecast instruction will do just that. In my case, I have data until 2000 and no data after 2000. So, for out of sample forecasts after 2000 I don't need a rolling regression and a one time forecast instruction should be enough to get out of sample data after 2000. I can then use the forecasted series prior and after 2000 to do backtesting etc.

Can you please confirm that my understanding is correct ?

[moderator]

I thought I could use rolling regressions in RATS with forecasted values but there is no need to do so since the forecast instruction will do just that.

Dynamic forecasting is not the same thing as re-estimating the model at each step.

If the model is purely endogenous (other than the constant), you can indeed FORECAST the model out of sample as many steps as desired, but all of those forecast steps will be computed using the same set of coefficients--the ones from your last estimation.

If you want to continue the rolling regression out of sample (using previous forecasts as "actual" values for the out-of-sample periods), you can do that by storing or copying the forecasted values into the actual endogenous variables.

Regards,
Tom Maycock

[apollon]

True. Can you pls give an example of how can I do that (I assume it has to be inside the loop) or is such an example mentioned in the user guide ?

[TomDoan]

What Meese and Rogoff are doing is a set of rolling regressions, and, at each step of that, doing dynamic forecasts for 1, 3, 6 and 12 steps. They then analyze those forecasts further. You seem to be trying to do moving estimation intervals, while Meese and Rogoff are adding data to the end while leaving the start period fixed. As you wrote this originally, you were using only 41 data points with a six variable, four lag VAR which is way too small. If you're trying to reduce the calculation time while debugging your calculations, do it by reducing the number of periods over which you loop, not by making your estimation range too small to be useful.

Code: Select all

system(model=varmodel)
* le is the logarithmic exchange rate
variables le sharediff idiff infdiff ydiff cadiff
lags 1 to 4
det constant
end(system)

To get the same types of forecast series as Meese and Rogoff (but with a moving window for estimation), you would do:

Code: Select all

do iend = ibegin,iend
   estimate(noprint) iend-<<bigger number than 40>> iend
   forecast(model=varmodel,results=forecasts,from=iend+1,to=iend+12,print)
   set fore1 iend iend = forecasts(1)(iend+1)
   set fore3 iend iend = forecasts(1)(iend+3)
   set fore6 iend iend = forecasts(1)(iend+6)
   set fore12 iend iend = forecasts(1)(iend+12)
end do iend

Now, the actual data for comparison with fore1(t) is in le(t+1), for fore3(t) in le(t+3), etc. The no-change forecast for comparision with any of the four forecast series is le(t).

[apollon]

Hi Tom

I didn't post the original topic of this thread but I followed up on it since it relates to my query. Now, Tom's reply was
"If you want to continue the rolling regression out of sample (using previous forecasts as "actual" values for the out-of-sample periods), you can do that by storing or copying the forecasted values into the actual endogenous variables."

How do I copy a forecasted value into the actual endogenous variable ? Assuming a simple 2 variable VAR system with factors a and b, it would probably be something like below though not correct:

system(model=varmodel)
variables a b
lags 1
det constant
end(system)

do fend=fstart,fend
estimate(noprint) fend-50 fend
forecast(model=varmodel1,results=forecasts,from=fend+1,to=fend+1,print)
compute a(1) = forecasts(1)(fend+1) *can u pls correct ?
compute b(1) = forecasts(2)(fend+1) *can u pls correct ?
end do fend

[TomDoan]

Hi Tom

I didn't post the original topic of this thread but I followed up on it since it relates to my query. Now, Tom's reply was
"If you want to continue the rolling regression out of sample (using previous forecasts as "actual" values for the out-of-sample periods), you can do that by storing or copying the forecasted values into the actual endogenous variables."

How do I copy a forecasted value into the actual endogenous variable ? Assuming a simple 2 variable VAR system with factors a and b, it would probably be something like below though not correct:

I'm a bit confused about what you want, as it sounds as if you want to do rolling regressions beyond the end of the data. That doesn't really make any sense---forecasts aren't at all similar to the other data points. If you were doing a series of rolling regressions with a fixed start point, the estimates wouldn't change at all when you add forecasts in, since the forecasts are fit perfectly by the previous period's estimates.