Bayesian VAR Forecasting

[TomDoan]

If it's an OLS VAR, then you can use FORECAST.

[AhmedSahlool]

and I keep Simulate with the BVAR model, or I change it it also to Forecast, so I can compare the two models.

[TomDoan]

You can't use FORECAST with the BVAR as you've set it up---that's why you use simulation methods. You can use FORECAST with the BVAR if you use the (somewhat unrealistic) assumptions used in Litterman's original technique, and you can use FORECAST with an OLS VAR. You could use simulation methods for computing the forecasts in the OLS VAR, but there's no reason to do that, since that would just be using simulation methods to estimate a mean that you can compute exactly.

[AhmedSahlool]

Dear Tom,

I hope this finds you well,

I would like to know if it's normal to have relative RMSE of two models; classical VAR and BVAR estimated with Gibbs sampling, very close to one?

Does it means that the BVAR forecasting ability is close to that of classical VAR?

Could it be a miss-specification of the BVAR model?

Thank you

[TomDoan]

There usually is a BVAR which does somewhat better. Are you saying it's close to 1 relatively uniformly. If that's the case, I would think you have far too loose a prior. Did you try doing this using standard Theil U analysis to pick the BVAR setup? You can test models quite a bit easier that way.

[AhmedSahlool]

Dear Tom,

I hope this finds you well,

I followed the example 7.8 Using RUNTHEIL to pick the BVAR setup, and I ended up with a BVAR model that performs better than OLS one, according to the statistics computed by RUNTHEIL.

However, as you know, I would like to compare the forecasting performance of BVAR model estimated with GIbbs sampling to a classic VAR. So, I first tried to get the same results (forecasting statistics) that I got with RUNTHEIL for classic VAR. I estimated the VAR model -I tried to estimate with and without the specify option-, then I used FORECAST (with Static option) to get the forecasts, however, I get different forecasting errors.

So I would like to know, how to use FORECAST to get the same results as with RUNTHEIL for a classic VAR model.

Then, how to do the same with Simulate for the BVAR estimated with Gibbs sampling?

Another question, do you approve the following loop to get the RMSE for any forecasting results, I get very different results than those of RUNTHEIL:

Do i=1,nvar
Set forecastEorrder0(i) fstart fend = (%modeldepvars(SBVAR_EG)(i)- Clas_simresults(i))**2
Sstats(mean) fstart fend forecastEorrder0(i)>>SSE0(i)
Compute RMS0(i) = sqrt(SSE0(i))
Display RMS0(i)
End do i

I have a concern about the Minnesota mean prior, could I use an average of (-0.2) for a variable?

Thank you very much

[TomDoan]

Dear Tom,

I hope this finds you well,

I followed the example 7.8 Using RUNTHEIL to pick the BVAR setup, and I ended up with a BVAR model that performs better than OLS one, according to the statistics computed by RUNTHEIL.

However, as you know, I would like to compare the forecasting performance of BVAR model estimated with GIbbs sampling to a classic VAR. So, I first tried to get the same results (forecasting statistics) that I got with RUNTHEIL for classic VAR. I estimated the VAR model -I tried to estimate with and without the specify option-, then I used FORECAST (with Static option) to get the forecasts, however, I get different forecasting errors.

You most definitely do not use the STATIC option on FORECAST. The whole idea is to do simulated out-of-sample forecasting.

So I would like to know, how to use FORECAST to get the same results as with RUNTHEIL for a classic VAR model.

The fourth case of RUNTHEIL in Example 7.8 is basically an OLS VAR.

Then, how to do the same with Simulate for the BVAR estimated with Gibbs sampling?

That's extremely complicated. There's no Kalman filter to roll the estimates forward, and there's no closed form expression for the forecast, so those have to be done using the mean or median of simulations. Get your model selected first. The difference between the Theil U's using Gibbs sampling and those using the Litterman method isn't going to be all that large.

Another question, do you approve the following loop to get the RMSE for any forecasting results, I get very different results than those of RUNTHEIL:

Do i=1,nvar
Set forecastEorrder0(i) fstart fend = (%modeldepvars(SBVAR_EG)(i)- Clas_simresults(i))**2
Sstats(mean) fstart fend forecastEorrder0(i)>>SSE0(i)
Compute RMS0(i) = sqrt(SSE0(i))
Display RMS0(i)
End do i

The Theil U table computes a separate RMSE for each forecast horizon. You appear to be doing a combined RMSE for all steps.

I have a concern about the Minnesota mean prior, could I use an average of (-0.2) for a variable?

Could you? Yes. I'm not sure I've ever seen anyone use a first lag prior mean that's negative.

[sanjeev]

I am using BVAR method to forecast FPISMREV series where the estimation period is 1995:10 to 2011:10 and the forecast evaluation period is 2011:11 to 2013:06. For calculating 1 step ahead forecasts the no. of observations available should be 20, corresponding to the period 2011:11 to 2013:06 (monthly data), however, in the output file, in the column for N Obs it displays 21. I cant understand the difference.




I am using the following program:

OPEN DATA "C:\Documents and Settings\Administrator\Desktop\BVAR.xlsx"

CALENDAR(M) 1995:4

DATA(FORMAT=XLSX,ORG=COLUMNS) 1995:04 2014:05 BSE FPISMOOTH NEERE GIIPIND GIIPOECD MSCIEM CORIW VOLNNEW FR4 FRISKQ FRISKQ1 VAREM VARIND STDIND II VOLNEERG ECRI FR4 FPISMREV VOLNNEW

procedure runtheil

option choice type 1 symmetric general

option real tightness 0.4

option real decay 1

option real other 0.7

local integer time

system(model=bvar)

variables fpismrev bse neere msciem ii giipind volnnew coriw

lags 1 to 2

det constant ecri

specify(mvector=||1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0||, type=symmetric, lagtype=harmonic, tightness=.4, decay=1) .7

end(system)

theil(setup,model=bvar, steps=12, from=2011:11, to=2013:06)

estimate(noprint) 1995:10 2011:10

theil

do time=2011:11;2013:06

Kalman

theil(noprint) time

end do time

theil(dump)

END

@runtheil(tightness=.4,type=symmetric,decay=1,other=.7)





Output

Step Mean Error Mean Abs Err RMS Error Theil U N Obs

Forecast Statistics for Series FPISMREV

1 0.5296142 2.3545205 3.3586390 0.7423 21

2 0.4871747 2.7074045 3.7367750 0.7918 20

3 0.5849315 3.0244701 3.9841466 0.7942 19

4 0.7880696 3.1733549 4.3475209 0.7823 18

5 -0.1247608 2.4454828 3.4966014 0.7286 17

6 -0.0599990 2.4809969 3.5827462 0.7259 16

7 0.2346258 2.4087405 3.5535487 0.8381 15

8 0.3816739 2.4584368 3.6576797 0.8023 14

9 0.7945417 2.5561706 3.8225806 0.7142 13

10 0.7914947 2.6646355 3.9474210 0.8506 12

11 1.1241456 3.0861086 4.1426950 0.8533 11

12 0.8590976 2.9805915 4.1752787 0.9676 10


Thanks,
Sanjeev

[TomDoan]

The first THEIL in this (after the ESTIMATE) will do forecasts using data through 2011:10. You're not counting that.

estimate(noprint) 1995:10 2011:10
theil
do time=2011:11;2013:06
Kalman
theil(noprint) time
end do time
theil(dump)
END

[sanjeev]

Thank You Sir, this was quite helpful. Further, how should i change the set of commands, so that my estimations of the bvar model are for the period 1995:10 to 2011:10 and forecast evaluation is strictly for the period 2011:11 to 2013:06 (not including 2011:10 in the forecasts)

Reference:

estimate(noprint) 1995:10 2011:10
theil
do time=2011:11;2013:06
Kalman
theil(noprint) time
end do time
theil(dump)
END
@runtheil(tightness=.4,type=symmetric,decay=1,other=.7)

Thanks,
Sanjeev

[TomDoan]

estimate(noprint) 1995:10 2011:10
theil
do time=2011:11;2013:06
Kalman
theil(noprint) time
end do time
theil(dump)
END
@runtheil(tightness=.4,type=symmetric,decay=1,other=.7)

Get rid of the TIME option that's colored red---that requests forecasts starting at TIME, which isn't what you want, since TIME is the end of the estimation period. Also, do you want the DO TIME to run through 2013:6. As written, that will give 2013:6 as the end of the estimation period, so the final THEIL will do forecasts starting 2013:7.

[sanjeev]

Sir, I tried the code without TIME option, but when I print the forecasts I get them from 2011:12 onwards (instead of 2011:11) (file attached).

estimate(noprint) 1995:10 2011:10
theil
do time=2011:11;2013:06
Kalman
theil(print)
end do time
theil(dump)
END
@runtheil(tightness=.4,type=symmetric,decay=1,other=.7)

Also, do you want the DO TIME to run through 2013:6. Also, do you want the DO TIME to run through 2013:6. As written, that will give 2013:6 as the end of the estimation period, so the final THEIL will do forecasts starting 2013:7.

Sir, I basically intend to obtain recursive forecasts. i.e. estimate the model till 2011:10 and get forecasts from 2011:11 to 2013:06, then estimate the model again till 2011:11 and get the forecasts from 2011:12 to 2013:06 ....and so on till .. I estimate the model till 2013:05 and obtain the forecast for 2013:06. I do not intend to look at the forecasts for 2013:07 and period beyond that.

Thanks,
Sanjeev

[TomDoan]

You don't have a PRINT option on the first THEIL instruction (right after the ESTIMATE). That's the one that does forecasts starting in 2011:11.

[sanjeev]

Thank You Sir !!