Neural Network Model

[John_Val]

Hello,
I am trying to estimate a neural network model with 8 hidden units and 4 inputes, for yen series. When I display the vector of parameters I am only getting 8 values. Shouldn't there be a total of 49 parameter values.

Secondly, for the nntest command, I ran the model over the period 144 244. For this period does it keep the parameter values constant that it got from nnlearn over period 1 144. Is nntest identical to the prj command for linnear models?

set DYEN = yen-yen{1}
nnlearn(save=mem,hidden = 8,nodirect,squash=logistic,cvcrit=0.0040,iters=100000,notrace) 1 144
# dyen{1 to 4}
# dyen
The weights converged after 49402 epochs
Mean Squared Error = 8.999987e-03

dis mem
1001.00000 4.00000 8.00000 1.00000 0.00000 1.00000 -4.23000 2.33000
-4.23000 2.33000 3.61465 6.17600 5.64131 -10.45427 -9.02658 1.20957 2.84210
3.07444

nntest 144 244 mem
# dyen{1 to 4}
# dyenO

[TomDoan]

Regarding the second question, yes, it uses the estimated values from the first sample. For a couple of examples, check out tsayp180.prg and tsayp181.prg.

I don't know what might be happening in the first case. I ran basically the same model and got the memory vector with the correct number of elements. If you're still having a problem, you should send the program and data to support@estima.com.

[John_Val]

Hello,
I'm going to send the program to estima, I still can't get print the full parameter set.
I have two more questions regarding NN model.
I estimated the model over period 1 144 and then used NNtest to obtain fitted values over the same period. I calculated the Mean Squared Error to be 0.7999 which is different then the MSE calcualed by NNLearn 0.0119769.

nnlearn(save=mem,hidden = 8,nodirect,squash=logistic,cvcrit=0.0120,iters=100000,notrace) 1 144
# dyen{1 to 4}
# dyen
The weights converged after 16 epochs
Mean Squared Error = 1.197696e-02

nntest 1 144 mem
# dyen{1 to 4}
# dyenO

com [real] error = 0
do i = 6,144
com error = error + (dyen(i) - dyenO(i))**2
end do i
com MSE = error / 138
dis MSE
0.79992

My second question is regarding, one step ahead forecasting. Are you able to loop over NNtest and NNlearn? Becasue I get the following error message when I try to generate 10 one step ahead forecasts:

set forcast = 0.0
do i =1,10
nnlearn(noprint,save=mem,hidden = 8,nodirect,squash=logistic,cvcrit=0.0120,iters=100000,notrace) i 144+i
# dyen{1 to 4}
# dyen
nntest i 145+i mem
# dyen{1 to 4}
# dyenO
## SX11. Identifier DYENO is Not Recognizable. Incorrect Option Field or Parameter Order?
>>>># dyenO<<<<

com forcast(145+i)=dyenO(145+i)
end do i

[TomDoan]

The mean squared errors used internally by NNTEST are standardized. That's what allows multiple target variables.

To fix the second problem, add

DECLARE SERIES DYENO

outside the loop. You can't introduce a new series on a supplementary card inside a compiled section like a DO loop.

[John_Val]

What factor do I use to standarize the mean squared errors?

And at each run through the loop does nnlearn use parameter values estimated on the previous run as starting values. If not, how can I make this happen?

[moderator]

I'll have to check to see exactly how the standardized MSE is computed.

See the descriptions of the SAVE and RESTART options in the NNLEARN section of the Reference Manual (and Using NNLEARN in Section 12.6 of the User's Guide) for details on the choices available for handling starting values for NNLEARN.

[TomDoan]

What factor do I use to standarize the mean squared errors?

Both inputs and outputs are first mapped linearly into the range [0,1] (if you use the default SQUASH=LOGISTIC). That's done with

R(t)=(X(t)-Xmin)/(Xmax-Xmin)

where X is the original series and R is the rescaled series. That's done separately for each input and each output. If you use the PAD option (to allow forecasts for a broader range of guess values), it maps the data to [pad/2,1-pad/2].

The mean squared errors reported are computed using the rescaled data so they would be divided by (max-min)^2 relative to the natural scale of the data.

[JohnV]

Hi,
Following the above posts I made about 2 years ago, I have restarted work on ANN and I have some more related questions.

I ran a 2 hidden layer model, with two lags of changes in SP500.

set dsp 5 200 = spp - spp{1}
nnlearn(noprint,save=mem,hidden = 2,nodirect,squash=logistic,cvcrit=0.0120,iters=100000,notrace) 5 200
# dsp{1 to 2}
# dsp

the memory vector mem should have in total 10 + 9 values. Yet when I display the vector I get 27 values.

1001.00000 2.00000 2.00000 1.00000 0.00000 1.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 -94.61207 93.02038 49.01557 -44.72746 67.46444 -95.19158 0.27903 0.23054 -23.09614 -34.17000 26.84000 -34.17000 26.84000 -34.17000 26.84000

It seems that the first 12 are not parameter estimates and the last 6 are just repeating themselves.

Assuming only -94.61 to -23.0914 are proper estimates, is the correct fitted values equal to

fit = -94.61 + 93.02*logistic(u1) +49.01*logistic(u2) where u1=-44.72 + 67.46*dsp{1} -95.19*dsp{2} and so on for u2

The reason I want to compute fitted values in stead of using nntest is becuase I want eventually plot the individual logistical functions to see the different type of frequencies in the data.

[moderator]

See the "Key to Memory Vectors" section under NNLEARN in the Reference Manual. It describes all the terms that are saved in the memory vector, many of which are flags or counters, not estimated parameters.

Regards,
Tom Maycock
Estima