DLM in Kalman filtering estimation and the EXACT, NEW Q

[Jennylai]

I am now using the DLM to esimate a Kalman filtering process with observations to be 1-dimension and state variables 3by1 dimension. Below is my programme, I am using the EXACT option in DLM so that Koopman(1997)'s exact initialization could be used.

* input data file
calendar(q) 1996:1
all 49
open data BOP_Q_MUSD.xls
data(format=xls,org=columns) / fa_usd

* unit root test
@dfunit(lags=4) fa_usd


*initialize the variables and matrices
nonlin v1 v2 v3
frml M3 = v1/sqrt(1+v1*v1)
frml Q1 = exp(v2)
frml Q3 = exp(v3)


dec rect z
dec vect x0
dec frml[rect] M
dec frml[symm] Q
dec frml[symm] sx0
dec frml[vect] f
dec frml[symm] sh0

compute z=||1.0|0.0|1.0||
compute x0=||0.0|0.0|0.0||
frml M = ||1.0,1.0,0.0|0.0,1.0,0.0|0.0,0.0,M3||
frml Q = ||Q1,0.0,0.0|0.0,0.0,0.0|0.0,0.0,Q3||
frml sx0 = ||0.0,0.0,0.0|0.0,0.0,0.0|0.0,0.0,Q3/(1-M3*M3)||
frml sh0 = ||1e7,0.0,0.0|0.0,1e7,0.0|0.0,0.0,0.0||
frml f = ||fa_usd||

* Kalman filter and parameter estimation
compute v1=v2=v3=0.0

dlm(trace,method=bfgs,a=M,c=z,sv=1.0,sw=Q,x0=x0,sx0=sx0,exact,sh0=sh0,y=f,variance=concentrated,vhat=vt,svhat=gt) / xt pt

My problem is that, whenever I ran this programme, a message always came up saying
"# DLM5. Probable Model Error. Diffuse prior was not reduced to zero rank"
May I ask how to deal with this problem?

Actually I deduced by hand that the kalman filter system I am using here could produce zero rank diffuse prior P(infinity).

Thank you! I need your help!

[TomDoan]

You can replace

Code: Select all

dlm(trace,method=bfgs,a=M,c=z,sv=1.0,sw=Q,x0=x0,sx0=sx0,exact,sh0=sh0,y=f,variance=concentrated,vhat=vt,svhat=gt) / xt pt

with

Code: Select all

dlm(trace,method=bfgs,a=M,c=z,sv=1.0,sw=Q,g=||0.0,0.0,1.0||,var=concentrated,y=f,vhat=vt,svhat=gt) / xt pt

which lets the program do most of the work of figuring out how to handle the mix of diffuse and non-diffuse elements in the initial vector. RATS isn't properly handling the accounting for the rank of the SH0 matrix (hence the warning message). We'll have to fix that, though use of SH0 is not recommended, since there are several much simpler ways of handling that. (G=||0.0,0.0,1.0|| or PRESAMPLE=ERGODIC).

[Jennylai]

Dear TomDoan,

Thank you very much for your excellent suggetions! I'll try it first!

Thank you very much for your kind help!

[Jennylai]

Hi TomDoan,

May I ask if RATS could automatically figure out the initial P(infinity) and P(star) that constitute the initial covariance matrix P, without being provided with the large Kappa(1e7) from us?

Thanks!

[TomDoan]

That's the whole point of the "G" option. That allows the DLM instruction to separate the diffuse (linear combinations of) states from the stationary ones. If you use that, you don't need the SH0, SX0 or X0 options, since all of those can be determined from the model. How this is done is described in the User's Guide.

By the way, if you use the SH0 option, just use 1's, not 1.0e+7's. The "infinity" multiplier on that is implied, so you don't have to approximate it.

[Jennylai]

Hi TomDoan,

Thank you very much for your reply!
Actually I have another question regarding this EXACT procedure in RATS. I found out that RATS is using the formula in Harvey(1989), which is slightly different from the fomula used in Durbin and Koopman(2001). And in Harvey(1989) and in RATS formula, the prediction and updating process are different, which means at/t-1 is not equal to at, and Pt/t-1 is not equal to Pt.

However, if I use the exact initialization for this Harvey(1989) version of state space model (also the RATS version as it is writtin in the User's guide), after the P(infinity,d+1)(d stands for the number of non-stationary series in the state variables) become ZERO, then there is no difference between prediction and updating, so at/t-1 is equal to at, Pt/t-1 is equal to Pt. May I ask if this is the process that RATS works with exact initialization? IF no difference exist between predicting and updating, are we losing important informations whenever we bring in a new observation on Yt?

Thank you very much!

[TomDoan]

The nice thing about the diffuse prior is that it is very forgiving. Assuming that you feed in an SH0 correctly (again, you're usually better off using the G option and letting RATS do the hard part), the first step (in the RATS/Harvey scheme) is to update this to A SH0 A'. While, in general, this will be different from SH0, as long as it spans the same space, it doesn't matter. Scaling and rotations on the diffuse prior matrix don't affect the results. (If done correctly), the rank only reduces when you have data to resolve the diffuseness, which will happen at the same time in either scheme.

[Jennylai]

Hi TomDoan ,

Thank you for your kind reply! May I ask another question? In RATS, as we use the option " concentrate", the variance (for example, if h=1.0) of the measurement equation will be concentrated out of the likelihood function. Then how would RATS perform the test on the estimate result of this variance? How can we know that this estimated variance is significant or not?

Thank you!