Simplex method in NLSYSTEM

[xin_zh]

Hi Tom,

I am using the NLSYSTEM instruction to estimate a system of nonlinear moment conditions. However, I could not get converged estimates, or converge but does not make sense and zero standard errors, or got error messages like "subiterations limit exceeded". I read through chapter 4 of the user's guide (UG), and found my problems are talked about in the troubleshooting section. It suggests in the troubleshooting section that I should try using preliminary simplex iterations first to get better initial guesses. In UG-116 I found that the simplex method is an option in NLSYSTEM. However, when I implemented "pmethod=simplex, piters=5" in the option of NLSYSTEM, the error message showed I could not use them in this instruction. Also, in the NLSYSTEM session in the reference manual (RM) starting from RM-326, I could not find the option "pmethod" and "piters". Could you please let me know how I can get better initial guesses when using NLSYSTEM?

Thanks,
Xin

[TomDoan]

The multivariate Gauss-Newton algorithm used by NLSYSTEM is much more stable numerically than (say) the BFGS used by MAXIMIZE. If you're having a problem with GMM, it's usually because the weight matrix won't converge. See page UG-139 in the v8 User's Guide.

[xin_zh]

Dear Tom,

Thank you very much for your reply! I tried to use "robusterrors" in stead of "zudep" as suggested on UG-139, however, I still could not get estimation results. As suggested on UG-152, "almost all problems are due to "bad" initial guess values". So I tried many different initial values, but none of them succeeded. And that was very time consuming, so I decided to use loop to try different initial values. The codes are:

Code: Select all

calendar(q) 1986:2
open data "C:\Users\Xin\Dropbox\0000\0703.xls"
*
data(format=xls,org=columns) 1986:2 2012:3 FUTPrice CASHPrice CommShortLag1 offfarmStock Sale StockC PPIiv tiv


*
nonlin Disct TrC RAa RAb RAc

frml h1 = Disct*exp(RAa*((CASHPrice(t+1)-TrC*CASHPrice(t+1)*Sale(t+1)/(Sale(t+1)+offfarmStock(t+1)))*Sale(t+1)$
          -StockC(t+1)+(CASHPrice(t+1)-FUTPrice(t+1))*CommShortLag1(t+1))+RAb*(((CASHPrice(t+1)-TrC*CASHPrice(t+1)*Sale(t+1)/(Sale(t+1)+offfarmStock(t+1)))*Sale(t+1)$
          -StockC(t+1)+(CASHPrice(t+1)-FUTPrice(t+1))*CommShortLag1(t+1))^2)+RAc*tiv(t+1))*((CASHPrice(t+1)-TrC*CASHPrice(t+1)*Sale(t+1)/(Sale(t+1)+offfarmStock(t+1)))$
          *0.97-TrC*CASHPrice(t+1)*offfarmStock(t+1)*Sale(t+1)/(0.97*offfarmStock(t)^2))-exp(RAa*((CASHPrice(t)-TrC*CASHPrice(t)*Sale(t)/(Sale(t)+offfarmStock(t)))*Sale(t)$
          -StockC(t)+(CASHPrice(t)-FUTPrice(t))*CommShortLag1(t))+RAb*(((CASHPrice(t)-TrC*CASHPrice(t)*Sale(t)/(Sale(t)+offfarmStock(t)))*Sale(t)$
          -StockC(t)+(CASHPrice(t)-FUTPrice(t))*CommShortLag1(t))^2)+RAc*tiv(t))*(CASHPrice(t)-TrC*2*CASHPrice(t)*Sale(t)/(Sale(t)+offfarmStock(t))+0.09)


frml h2 = exp(RAa*((CASHPrice(t+1)-TrC*CASHPrice(t+1)*Sale(t+1)/(Sale(t+1)+offfarmStock(t+1)))*Sale(t+1)$
          -StockC(t+1)+(CASHPrice(t+1)-FUTPrice(t+1))*CommShortLag1(t+1))+RAb*(((CASHPrice(t+1)-TrC*CASHPrice(t+1)*Sale(t+1)/(Sale(t+1)+offfarmStock(t+1)))*Sale(t+1)$
          -StockC(t+1)+(CASHPrice(t+1)-FUTPrice(t+1))*CommShortLag1(t+1))^2)+RAc*tiv(t+1))*(CASHPrice(t+1)-FUTPrice(t+1))


compute [vector] Aa=%seqa(-0.001,0.001,1000)
do i=1,1000
	compute Disct = 0.99,TrC = 0.0001,RAa =Aa(i), RAb = 0.001, RAc = -0.0000001
	instruments constant PPIiv  CASHPrice{1 to 6}
	nlsystem(inst,robusterrors, iterations=1000, cvcrit=0.0001) / h1 h2
end do i

However, I got no results. And the "Aa" series does not show up in the "series window". Could you please give me some suggestions? Thank you very much! Data are in the attachment.

Thanks,
Xin

[TomDoan]

Your problem is that the two functions are effectively uncomputable at your guess values. If you do

set test1 = h1
set test2 = h2

after you've set your guess values, you'll find that the results aren't usable (numbers like 1+40).

There is either something wrong with the formulas, or your guess values are way off. I assume you really mean to be using future variables (t+1) subscripts in your two functions. You could simplify it considerably using "subformulas" for calculations like (CASHPrice(t+1)-FUTPrice(t+1))*CommShortLag1(t+1) which gets used quite a few times.

If you do

frml disc = (cashprice-futprice)*commshortlag1

then you can use disc(t+1) where you need (CASHPrice(t+1)-FUTPrice(t+1))*CommShortLag1(t+1) and disc(t) where you need (CASHPrice(t)-FUTPrice(t))*CommShortLag1(t)

If you clean up those repeated calculations, you might be able to pick up where you (most likely) have some type of formula error.

[xin_zh]

Hi Tom,

Thank you so much for your quick response! I highly appreciate your help. I will do as you suggested.

Have a good evening,
Xin

[xin_zh]

Dear Tom,

As you suggested, I set "subformulas" and cleaned up the calculation. And before run the estimation, I first calculate the moment conditions to see if they are computable. Then I do the estimation with the initial values when the function values seem computable. The nonlinear GMM is estimated using both "zudep" and "robusterrors". However, "zudep" always gives no-converge and "robusterrors" gives error message "## REG12. SIGMA Is Singular/Not PSD At Row 0. Too Many Equations for Data Set Size?" The codes are:

Code: Select all

calendar(q) 1986:2
open data "C:\Users\Xin\Dropbox\Farmer\Commercial1\Combine.xls"
*
data(format=xls,org=columns) 1986:2 2012:3 FUTPrice CASHPrice CommShortLag1 offfarmStock Sale StockC PPIiv tiv

frml Temp1 = CASHPrice*Sale/(Sale+offfarmStock)
frml Temp2 = (CASHPrice-FUTPrice)*CommShortLag1

*
nonlin Disct TrC RAa RAb RAc

frml h1 = Disct*exp(RAa*((CASHPrice(t+1)-TrC*Temp1(t+1))*Sale(t+1)-StockC(t+1)+Temp2(t+1))+RAb*(((CASHPrice(t+1)-TrC*Temp1(t+1))*Sale(t+1)-StockC(t+1)+Temp2(t+1))^2)+RAc*tiv(t+1))$
		*((CASHPrice(t+1)-TrC*Temp1(t+1))*0.97-TrC*CASHPrice(t+1)*offfarmStock(t+1)*Sale(t+1)/(0.97*offfarmStock(t)^2))$
		-exp(RAa*((CASHPrice(t)-TrC*Temp1(t))*Sale(t)-StockC(t)+Temp2(t))+RAb*(((CASHPrice(t)-TrC*Temp1(t))*Sale(t)-StockC(t)+Temp2(t))^2)+RAc*tiv(t))$
		*(CASHPrice(t)-TrC*2*Temp1(t)+0.09)


frml h2 = exp(RAa*((CASHPrice(t+1)-TrC*Temp1(t+1))*Sale(t+1)$
          -StockC(t+1)+Temp2(t+1))+RAb*(((CASHPrice(t+1)-TrC*Temp1(t+1))*Sale(t+1)$
          -StockC(t+1)+Temp2(t+1))^2)+RAc*tiv(t+1))*(CASHPrice(t+1)-FUTPrice(t+1))

***********************************************************************

compute [vector] Aa=%seqa(-0.0005,0.0001,50)
do i=1,50
	compute Disct = 0.7,TrC = 0.1,RAa = -0.01, RAb = -0.001, RAc = Aa(i)
	set test1 = h1
	set test2 = h2
	print / test1 test2
end do i

*************************************************************************
compute [vector] Aa=%seqa(-0.0005,0.0001,50)
do i=1,50
	compute Disct = 0.7,TrC = 0.1,RAa = -0.01, RAb = -0.001, RAc = Aa(i)
	instruments constant PPIiv  CASHPrice{1 to 4}
	nlsystem(inst,robusterrors,iterations=500,subiterations=500,cvcrit=0.00001) / h1 h2
end do i

I think this might still be the initial value problem. But finding the right initial values is like finding a needle in the ocean. Do you have any suggestions on my problem? I greatly appreciate your help! Thanks! Data are in the attachment. It's for the extended period from the data attached above, but the model is exactly the same.

Have a nice day,
Xin

[TomDoan]

Is this your model, or is this someone else's model applied to your data? I would be concerned about

frml Temp1 = CASHPrice*Sale/(Sale+offfarmStock)

as the values of Sale are sometimes negative. That means that sale+offfarmstock could be near zero, which could make for a large value of the exponent for certain data points. I'm also a bit concerned about the fact that the scale of the prices (are they your only two nominal variables?) don't seem to wash out of the calculations---you have terms which intermix linear and squared prices without a scale constant to correct for that.

[xin_zh]

Hi Tom,

Thanks for your quick response! I don't quite understand your concern about the nominal prices. Exactly, the two prices are only nominal terms in the model. Are there any problems with linear and quadratic prices in one equation?

I understand your concern on the negative sale. The model is developped by somebody else who got pretty nice results for a subperiod of the period in my data. We used the same data (or at least very similar because he collected his data back to his time and I collected my data recently) except for the length of time. I am looking for change before and after a time point. Therefore I added a time trend in the model which is "tiv". Interestingly, when I estimate the model separately (without the time trend variable) for the two periods, I got converged estimates. But I really want to add the time trend if I can.

Thanks for your consulting! I appreciate!
Xin

[TomDoan]

Hi Tom,

Thanks for your quick response! I don't quite understand your concern about the nominal prices. Exactly, the two prices are only nominal terms in the model. Are there any problems with linear and quadratic prices in one equation?

Think physics and making sure your units work out. Yours don't. Free coefficients can fix problems if used properly (one can have an interpretation of "per dollar" and another "per widget") but without them if you measure price in one unit, the square may dominate, and in another the linear may dominate. But your choice for units for the prices shouldn't affect real quantities.

I understand your concern on the negative sale. The model is developped by somebody else who got pretty nice results for a subperiod of the period in my data. We used the same data (or at least very similar because he collected his data back to his time and I collected my data recently) except for the length of time. I am looking for change before and after a time point. Therefore I added a time trend in the model which is "tiv". Interestingly, when I estimate the model separately (without the time trend variable) for the two periods, I got converged estimates. But I really want to add the time trend if I can.
Xin

If you want "before and after", you might want to think about a dummy. Putting a time trend inside an exp will require quite a bit of care getting the scale on it correct.

[xin_zh]

Hi Tom,

Thanks again for your replies! I understand your concern for prices now. I thought about this problem too, but since the magnitude of the prices are not big so I thought this might not be a problem. I also was wanting to use the dummy variable but my professor said it was a crude way to detect the change. It makes sense because things happened gradually, not immediately. Anyway I will keep working on this, and thank you very much for the help!

Have a good evening,
Xin

[TomDoan]

Hi Tom,

Thanks again for your replies! I understand your concern for prices now. I thought about this problem too, but since the magnitude of the prices are not big so I thought this might not be a problem. I also was wanting to use the dummy variable but my professor said it was a crude way to detect the change. It makes sense because things happened gradually, not immediately. Anyway I will keep working on this, and thank you very much for the help!

Have a good evening,
Xin

Start with a dummy. If you can get that to work, you might want to try some type of smooth transition, which has a limited range, but allows the change to be more gradual.

[xin_zh]

Thanks, Tom! I will do that!

Have a good night,
Xin