limits on sample size/variables

[moroche]

Hi Tom,
Is there any limit to the sample size that RATS can handle?
Or does this depend on the limitations of the computer?

I am starting some new research using very high frequency data (per second and per minute) and am trying to figure out if RATS can handle this amount of data.

Presuming that my PC (it has a quad core chip) can handle this data what should I set the calender command to be
cal(perday=1440,daily) 2000 1 2
all 2007:2:28//1440

regards,
Maurice

[moderator]

There is no hard limit on the sample size RATS can handle, up to the limitations of 32-bit addressing (or 64-bit addressing for 64-bit versions of RATS, which are currently only available for UNIX and Linux).

So, assuming you are using WinRATS or MacRATS, you are limited to accessing a a maximum of 2Gb of memory. Each real number requires 8 bytes of storage, so you can multiply your total number of observations by 8 to get a rough estimate of how much data you can handle.

For example, your CAL and ALL setup:

cal(perday=1440,daily) 2000 1 2
all 2007:2:28//1440

gives 2,689,920 observations per series, so you'll need roughly 22 Mb of RAM for each data series. Multiply that by the number of series you need to work with, and you'll have a ballpark estimate of the memory requirements.

Note that you also need to allow for temporary space for any calculations you need to do, storing reserved variables, some overhead for storing additional information about each series, and so on. But it works to give a rough idea.