I am running a long series of regressions reading the specification (ie which variables to include on the right hand side) from a matrix (it has several hundred thousand lines but about 60 columns). I am running out of memory. The software displays a message that: " ## M4. A memory request for an additional 100020 bytes cannot be satisfied ". I am not sure why this is happening since I am not creating any more series, but let's leave this on the side for now.
My latest thought is to write the matrix as a separate file on my disk and then have the programme read it one line at the time (so as not to have it in memory). I vaguely remember doing something like this with an ancient version of RATS meny years ago, but I could not find the answer in the current version of the manuals.
If there is a way of doing this could I actually have TWO of these files open at the same time? One which will give me the specificationn of the regression and another where I can store a few statistics from the output? Each pass of the loop will need to read from one and write to the other.
Finally, when I am all done I will need to find the minimum value of the first column of the results matrix. Does this mean that I have to import the entire matrix to memory (presumably creating more memory problems?
Any ideas and guidance would be appreciated.
Kostas
