evaluation of forecasts under VAR

[sanjeev]

In case the variables in the model have a unit root, following Sims (1980) and Doan (1990), we can estimate VAR in levels in case a cointegrating relationship exists. Then VAR will have variables with unit roots. Is it then possible to use the Modified DM test and the encompassing tests due to Clarke et. al (2001,2005)(for nested models) or do these tests work only in the absence of unit roots in the VAR. The papers that I have read use these tests for VAR with stationary variables.

In the case of BVAR, even if the variables are nonstationary, they can continue to be specified in levels in a BVAR model because as pointed out by Sims et. al (1990, p.136) ‘……the Bayesian approach is entirely based on the likelihood function, which has the same Gaussian shape regardless of the presence of nonstationarity, [hence] Bayesian inference need take no special account of nonstationarity’. Furthermore, Dua and Ray (1995) show that the Minnesota prior is appropriate even when the variables are cointegrated. Can we use the Clarke et. al tests(2001,2005) for nested BVAR models?

Are there any other tests apart for the encompassing tests literature that can be used to compare the forecasting performance of alternative VAR and BVAR models that are nested

[tclark]

The theory underlying the Clark and McCracken tests require least squares estimation and stationarity. In our nested model results, the distributions of the tests of equal accuracy or encompassing are driven entirely by the sampling error of the model parameter estimates, and deriving theoretical results is only tractable under the aforementioned assumptions. The tests will probably work fine with the model setup you describe, but our theory does not provide formal justification. If you produce the forecasts by estimating the models with rolling windows of data, the theoretical results of Giacomini and White (2006, Econometrica) provide justification for Diebold-Mariano tests of equal forecast accuracy applied to forecasts from the setup you describe. However, note that the null hypothesis appropriate for the Giacomini-White approach differs from the null under the Clark-McCracken approach: the former null is equal accuracy at estimated parameters, and the latter null is equal accuracy at the population level. This is discussed in more detail in the Clark-McCracken chapter in vol. 2 of the Handbook of Economic Forecasting (Elsevier, 2013).

[sanjeev]

Thanks a lot for the explanation.

[sanjeev]

Dear Tom, Thank you for your help. But ihave one more question.
I am using the following code to forecast the dlwheatfp. With three hidden layers, it is showing convergence. But I want to know how many nodes are there in each hidden layer.

nnlearn(rsquared=.10,iters=500000,hidden=3,direct,save=nnmodel) * 800
# dlwheatfp{basis1 ratiohightolow1 dlwheatfp1 dlgramfp1 drealroi1 dlnusafpr1}
# dlwheatfp
*
nntest 801 891 nnmodel
# dlwheatfp{basis1 ratiohightolow1 dlwheatfp1 dlgramfp1 drealroi1 dlnusafpr1}
# nnfore
@uforeerrors dlwheatfp nnfore
*
Thanks and regards,
Sanjeev

[TomDoan]

I'm not sure where you got the idea that the HIDDEN option gives the number of layers---there's one hidden layer and HIDDEN gives the number of nodes, and I can't see anywhere in the documentation where that isn't clear.