DLM estimation of hyper parameters

[Luckyboy]

Hi,
I am estimating a TVP model of the form
yt = B0t + B1xi1+B2t xi2+et
B0t = B0,t-1+ nt
B1t = B1,t-1+ psit
B2t = B2,t-1 + zetat
The estimation of the hyper parameters yields:
DLM - Estimation by BFGS
NO CONVERGENCE IN 39 ITERATIONS
LAST CRITERION WAS 0.0000097
ESTIMATION POSSIBLY HAS STALLED OR MACHINE ROUNDOFF IS MAKING FURTHER PROGRESS DIFFICULT.
TRY HIGHER SUBITERATIONS LIMIT, TIGHTER CVCRIT, DIFFERENT SETTING FOR EXACTLINE OR ALPHA ON NLPAR.
RESTARTING ESTIMATION FROM LAST ESTIMATES OR DIFFERENT INITIAL GUESSES MIGHT ALSO WORK
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 730.85960

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. SIGSQEPS -1.4710e-006 3.6827e-007 -3.99435 0.00006487
2. SIGSQETA 5.1012e-006 1.6252e-006 3.13884 0.00169619
3. SIGSQPSI 2.8818e-006 9.6535e-007 2.98522 0.00283377
4. SIGSQZETA -2.0521e-004 3.1602e-005 -6.49378 0.00000000
After diff guess value trial estimation, i later got


DLM - Estimation by BFGS
Convergence in 36 Iterations. Final criterion was 0.0000021 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 738.81085

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. SIGSQEPS -1.5369e-006 2.9357e-007 -5.23523 0.00000016
2. SIGSQETA 8.6127e-007 1.9050e-006 0.45210 0.65119590
3. SIGSQPSI 3.9445e-006 1.2344e-006 3.19558 0.00139548
4. SIGSQZETA 3.5895e-004 2.5009e-004 1.43531 0.15119804
The coeff of sigsqeps is negative, i tried zeroing it to zero but its not working out, pls what should i do?

[TomDoan]

If you look at:

http://www.estima.com/forum/viewtopic.php?f=5&t=631

the type of numerical problem you're having is handled by estimating the variances in log form. This isn't uncommon because the variances in TVP models tend to be quite small.

[Luckyboy]

Hi Tom,
Thank you very much for your suggestion.. pls, i have tried estimating the variances in log form but i noticed this error information.. could u please help me out?
Nonlin sigsqeps sigsqeta sigsqpsi sigsqzeta
stat(noprint) imp
compute sigsqeps =log(.001*%variance),sigsqeta = log(5.1012e-006)
compute sigsqpsi =log(2.8818e-006),sigsqzeta = log(3.6139e-004)
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = %diag(||sigsqeta,sigsqpsi,sigsqzeta||)
dec frml[rect] cv
frml cv = ||1.0,price, implag||
dlm(y=imp,a=av,c=cv,sv=exp(sigsqeps),sw=%diag(%exp(swv)),method=bfgs,presample=ergodic,type=filter) 1970:1 2009:3
## MAT6. Trying to Store 3 x 3 Matrix Into VECTOR
Thank you very much

[TomDoan]

Code: Select all

frml swv = %diag(||sigsqeta,sigsqpsi,sigsqzeta||)
dec frml[rect] cv
frml cv = ||1.0,price, implag||
dlm(y=imp,a=av,c=cv,sv=exp(sigsqeps),sw=%diag(%exp(swv)),method=bfgs,presample=ergodic,type=filter) 1970:1 2009:3

Take the %diag off the first line in this snippet. You need to leave the variances in vector form until after you do the %exp.

[Luckyboy]

Hi Tom,
Thank you very much for your help.. pls could you check these two output for me.. what is your opinion about the estmates?
(1)
DLM - Estimation by BFGS
Convergence in 3 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 156
Log Likelihood 667.99842

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. SIGSQEPS 7.26376e-006 8.81996e-007 8.23559 0.00000000
2. SIGSQETA 1.56400e-007 0.00000 0.00000 0.00000000
(2)
DLM - Estimation by BFGS
Convergence in 3 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 156
Log Likelihood 721.71997

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. SIGSQEPS 1.09468e-006 4.41919e-007 2.47711 0.01324497
2. SIGSQETA 1.75950e-006 0.00000 0.00000 0.00000000
Thanks

[TomDoan]

They don't look right. First off, the first of the two fits very poorly relative to the second. Also, both seem to be indicating that SIGSQETA doesn't even enter into the settings for the model (zero standard errors generally indicate that nothing done with that parameter effects the function value). You also still seem to be using the variances in their natural form rather than in logs, which is usually a bad idea with TVP models, because the values of the variances are so small.

[Luckyboy]

Hi Tom,
Thank you very much for all the troubles.. meanwhile, i have tried estimating the model variances in log form but still the estimates still suffer from numerical problem..what would you recommend for these two output?


linreg(noprint) imp
# constant price implag
compute sigsqeps=%seesq
linreg(noprint) imp
# constant price
compute sigsqpsi=%seesq
linreg(noprint) imp
# constant implag
compute sigsqzeta=%seesq
linreg(noprint) imp
# constant
compute sigsqeta=%seesq
Nonlin lsigsqeps lsigsqeta lsigsqpsi sigsqzeta
stat(noprint) imp
compute lsigsqeps =.01*%variance,lsigsqeta =.01*lsigsqeps
compute lsigsqpsi =3.9298,lsigsqzeta =3.6139e-004
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||lsigsqeta,lsigsqpsi,lsigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price,implag||
dlm(variance=concentrated,y=imp,a=av,c=cv,sv=1.0,sw=%diag(||lsigsqeta,lsigsqpsi,lsigsqzeta||),method=bfgs,exact,type=filter) 1970:1 2009:3

DLM - Estimation by BFGS
Convergence in 91 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 732.87010
Concentrated Variance 2.08788306e-025

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. LSIGSQEPS 4.39886e-006 0.00000 0.00000 0.00000000
2. LSIGSQETA 8.46151e+018 2.09085e+018 4.04693 0.00005189
3. LSIGSQPSI 9.95913e+018 2.44615e+018 4.07135 0.00004674
4. SIGSQZETA 0.00000 0.00000 0.00000 0.00000000


(2)Nonlin lsigsqeps lsigsqeta lsigsqpsi lsigsqzeta
stat(noprint) imp
compute lsigsqeps =log(.0001*%variance),lsigsqeta =log(5.1012e-006)
compute lsigsqpsi =log(2.8818e-006),lsigsqzeta = log(3.6139e-004)
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||lsigsqeta,lsigsqpsi,lsigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price, implag||
dlm(y=imp,a=av,c=cv,sv=exp(lsigsqeps),sw=%exp(%diag(||lsigsqeta,lsigsqpsi,lsigsqzeta||)),method=bfgs,presample=ergodic,type=filter) 1970:1 2009:3

DLM - Estimation by BFGS
Convergence in 31 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood -213.98361

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. LSIGSQEPS -27.17568 590.73278 -0.04600 0.96330758
2. LSIGSQETA -505.30284 28455.14903 -0.01776 0.98583201
3. LSIGSQPSI -451.60812 25323.77121 -0.01783 0.98577179
4. LSIGSQZETA -774.28518 44222.50280 -0.01751 0.98603067

[TomDoan]

Nonlin lsigsqeps lsigsqeta lsigsqpsi sigsqzeta
stat(noprint) imp
compute lsigsqeps =.01*%variance,lsigsqeta =.01*lsigsqeps
compute lsigsqpsi =3.9298,lsigsqzeta =3.6139e-004
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||lsigsqeta,lsigsqpsi,lsigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price,implag||
dlm(variance=concentrated,y=imp,a=av,c=cv,sv=1.0,sw=%diag(||lsigsqeta,lsigsqpsi,lsigsqzeta||),method=bfgs,exact,type=filter) 1970:1 2009:3

DLM - Estimation by BFGS
Convergence in 91 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 732.87010
Concentrated Variance 2.08788306e-025

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. LSIGSQEPS 4.39886e-006 0.00000 0.00000 0.00000000
2. LSIGSQETA 8.46151e+018 2.09085e+018 4.04693 0.00005189
3. LSIGSQPSI 9.95913e+018 2.44615e+018 4.07135 0.00004674
4. SIGSQZETA 0.00000 0.00000 0.00000 0.00000000

You still have those 0 standard errors. Where is LSIGSQEPS entering into the model? Nowhere. The variance of epsilon has been concentrated out. What about SIGSQZETA? It doesn't appear; it should now be LSIGSQZETA. Because you're not exping the log variances, the LSIGSQETA and LSIGSQPSI are estimating ratios to the standardizing variance (that for epsilon). Even though they look wrong (too big), they actually are probably reasonable. If you concentrate out a variance that's nearly zero, the remaining relative variances will be quite high.

(2)Nonlin lsigsqeps lsigsqeta lsigsqpsi lsigsqzeta
stat(noprint) imp
compute lsigsqeps =log(.0001*%variance),lsigsqeta =log(5.1012e-006)
compute lsigsqpsi =log(2.8818e-006),lsigsqzeta = log(3.6139e-004)
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||lsigsqeta,lsigsqpsi,lsigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price, implag||
dlm(y=imp,a=av,c=cv,sv=exp(lsigsqeps),sw=%exp(%diag(||lsigsqeta,lsigsqpsi,lsigsqzeta||)),method=bfgs,presample=ergodic,type=filter) 1970:1 2009:3

You have your %exp and %diag in the wrong order in the definition of SW. As written, you'll have 1's on the off diagonal, not zeros.

[Luckyboy]

Hi Tom,
I am really indebted to you for all your help.. May God Almighty reward you.. Meanwhile, i effected the wrong ordering in Sw as detected by you.. The result obtained still suffers from the numerical problem.. i.e
Nonlin lsigsqeps lsigsqeta lsigsqpsi lsigsqzeta
stat(noprint) imp
compute lsigsqeps =log(.0001*%variance),lsigsqeta =log(5.1012e-006)
compute lsigsqpsi =log(2.8818e-006),lsigsqzeta = log(3.6139e-004)
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||lsigsqeta,lsigsqpsi,lsigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price, implag||
dlm(y=imp,a=av,c=cv,sv=exp(lsigsqeps),sw=%diag(%exp(||lsigsqeta,lsigsqpsi,lsigsqzeta||)),method=bfgs,exact,type=filter) 1970:1 2009:3

DLM - Estimation by BFGS
Convergence in 44 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 736.19419

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. LSIGSQEPS -39.1958282 324.3217356 -0.12085 0.90380607
2. LSIGSQETA -33.6607758 335.8748401 -0.10022 0.92017109
3. LSIGSQPSI -13.1310796 0.1141763 -115.00708 0.00000000
4. LSIGSQZETA -7.8944034 0.0185595 -425.35647 0.00000000
pls, what can be done to correct for this negative coeffs (-39.195, -33.6607,-13.13, -7.89)?

secondly, to improve the behaviour of the estimates, i decided to try concentration of variance techniques as suggested in Durbin and Koopman (2001) "Time series by SPM"..
I reparameterise the model.i.e
et ...N(0,sigma**2) for the meaurement eqn and the following for the state equation ; nt..N(0,q*sigma**2), pt ..N(0,psi**2) and Qt ...N(0,zeta**2).
i used the code below and obtained the result that follows:

linreg(noprint) imp
# constant price implag
compute sigsqeps=%seesq
linreg(noprint) imp
# constant price
compute sigsqpsi=%seesq
linreg(noprint) imp
# constant implag
compute sigsqzeta=%seesq
linreg(noprint) imp
# constant
compute sigsqeta=%seesq
Nonlin psi lsigsqpsi sigsqzeta q
stat(noprint) imp
compute psi =0.0, q = exp(psi)
compute lsigsqpsi = 3.9298,lsigsqzeta =3.6139e-004
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||psi,lsigsqpsi,lsigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price,implag||
dlm(variance=concentrated,y=imp,a=av,c=cv,sv=1.0,sw=%diag(%exp(||psi,lsigsqpsi,lsigsqzeta||)),scale,method=bfgs,exact,type=filter) 1970:1 2009:3

DLM - Estimation by BFGS
Convergence in 18 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 732.87082
Concentrated Variance 7.86400557e-039

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. PSI 74.516366650 10.905520086 6.83290 0.00000000
2. LSIGSQPSI 74.640189534 10.616380396 7.03066 0.00000000
3. SIGSQZETA 0.000005172 0.000000000 0.00000 0.00000000
4. Q 1.000000000 0.000000000 0.00000 0.00000000

is the code for the estimation done by the method of concentration of variance Ok? All coeffs are now positive but the concentrated variance (ie variance of et seems to be very small)..
I would appreciate your comments on this.. Thanks

[TomDoan]

You still have problems with your choice of parameters---two of them don't appear in the model.

That's not that odd for the measurement equation variance to be (effectively) zero in a TVP model with that many coefficients; the data can often be fit quite well just through adjustments to the coefficients. In fact, it's often the case that allowing the variances to be estimated freely produces very unreasonable results, with far too much coefficient variation.

[Luckyboy]

Hello Mr Tom,
Thank you very much for all your invalauble comments so far.. After so many trials, i discovered that the log form estimation may not be appropriate because the variances are quite small.. I decided to concentrate the measurement error out of the estimation.. This two seems to be the best out of the many estimations i have carried out..
please, what is your comment as regards these two estimations below
(1)
Nonlin theta sigsqpsi sigsqzeta
compute theta=log(0.051)
compute sigsqpsi =log(sigsqzeta*0.1),sigsqzeta =log(sigsqzeta)
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||theta,sigsqpsi,sigsqzeta||
dec frml[rect] cv
frml cv = ||1.0,price,implag||
dlm(y=imp,a=av,c=cv,sv=1.0,sw=%diag(%exp(||theta,sigsqpsi,sigsqzeta||)),method=bfgs,exact,type=smooth,variance=concentrated) 1970:1 2009:3
****************SELECTED ESTIMATE************************************************
DLM - Estimation by BFGS
Convergence in 43 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 736.19419
Concentrated Variance 8.56547195e-068

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. THETA 129.01426315 5.94197328 21.71236 0.00000000
2. SIGSQPSI 141.29707113 0.89158184 158.47908 0.00000000
3. SIGSQZETA 146.53345690 0.91908389 159.43426 0.00000000


disp "sigsqeps" %variance
sigsqeps 8.56547e-068
disp "sigsqeta" %variance*theta
sigsqeta 1.10507e-065

(2)
Nonlin theta sigmapsi sigmazeta
compute theta =0.1
compute sigmapsi =sigsqpsi,sigmazeta =sigsqzeta*0.01
dec frml[rect] av
dec frml[symm] swv
frml av = ||1.0,0.0,0.0|0.0,1.0,0.0|0.0,0.0,1.0||
frml swv = ||theta,sigmapsi,sigmazeta||
dec frml[rect] cv
frml cv = ||1.0,price,implag||
dlm(variance=concentrated,y=imp,a=av,c=cv,sv=1.0,sw=%diag(||theta,sigmapsi,sigmazeta||),method=bfgs,exact,type=filter) 1970:1 2009:3

DLM - Estimation by BFGS
Convergence in 53 Iterations. Final criterion was 0.0000000 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 155
Log Likelihood 732.88414
Concentrated Variance 6.20234160e-011

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. THETA 28999.567417 1557.591519 18.61821 0.00000000
2. SIGMAPSI 33105.513939 3154.633165 10.49425 0.00000000
3. SIGMAZETA 12528.585922 818.442000 15.30785 0.00000000
disp "sigsqeps" %variance
sigsqeps 6.20234e-011
disp "sigsqeta" %variance*theta
sigsqeta 1.79865e-006

I am also thinking of using the filter ( either centered or Hp) to generate initial guess value instead of the linreg residuals and stats ..
Hope to hear from you..
Thanks

[TomDoan]

Those appear to be converging just fine; it's just that you're concentrating out a variance which is being estimated as (in effect) zero. However, you are estimating the ratios between the coefficient disturbance variances and the measurement variance in log form. So you want to use

disp "sigsqeta" %variance*exp(theta)

not

disp "sigsqeta" %variance*theta

Similarly for the others.

[Luckyboy]

hi tom,
"disp "sigsqeta" %variance*exp(theta)"

not

disp "sigsqeta" %variance*theta

Similarly for the others.

do you mean that when i reduced the dimension of the numerical search by concentration of one of the variance,
i used, sigmaeta=thetasigmaepsilon, in the model with 3 or more variances, does this reparameterization affects the others? i was wondering why u added, "Similarly for the others"
Thanks

[TomDoan]

I mean that all three of the other variances are estimated in log ratio form, so to get the actual variances you need

exp(estimated value)*%variance.

[Luckyboy]

Hi Tom,
please i got these results from my estimation.. the first was done when i estimate the data in their level, but the result was obtain when i multiply the data set by 100 prior to the estimation.. which of them will favourably be consider?

(1)
DLM - Estimation by BFGS
Convergence in 58 Iterations. Final criterion was 0.0000038 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 156
Log Likelihood 295.05661
Concentrated Variance 1.41990356e-015

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. THETA 1.50077e+011 7.52989e+009 19.93079 0.00000000
2. SIGSQPSI 9.64890e+010 4.84158e+009 19.92921 0.00000000
3. SIGSQZETA 4.93081e+009 2.47203e+008 19.94642 0.00000000

disp "sigsqeps" %variance
sigsqeps 1.41990e-015
disp "sigsqeta" %variance*theta
sigsqeta 2.13094e-004
disp "sigsqpsi" %variance*9.64890e+010
sigsqpsi 1.37005e-004
disp "sigsqzeta" %variance*4.93081e+009
sigsqzeta 7.00127e-006

(2)
DLM - Estimation by BFGS
Convergence in 33 Iterations. Final criterion was 0.0000020 <= 0.0000100
Quarterly Data From 1970:01 To 2009:03
Usable Observations 159
Rank of Observables 156
Log Likelihood -428.17342
Concentrated Variance 2.12571839e-011

Variable Coeff Std Error T-Stat Signif
********************************************************************************
1. THETA 8038.79 2671.84 3.00871 0.00262360
2. SIGSQPSI 240504969.55 79977517.78 3.00716 0.00263703
3. SIGSQZETA 283725.65 137473.66 2.06385 0.03903150

pls your comment will be appreciated..
Thanks