problem reading matlab data files

[Nabtheberliner]

Hi everyone,
I work on book Econometric Modelling with Time Series,specification, estimation and testing by Vance Martin,Stan HUrn, David Harris,Cambridge University Press.
I try to open the datasets with RATS8.0 in order to write the programs and everytime the variables are nameless and empty when i check the series window. It does the same with the xls data files. On the other hand, with another software like Jmulti it works, all the variables exist.
I attach some files here if someone wants to give it a try.
Thanks people.
²

[TomDoan]

Hi everyone,
I work on book Econometric Modelling with Time Series,specification, estimation and testing by Vance Martin,Stan HUrn, David Harris,Cambridge University Press.
I try to open the datasets with RATS8.0 in order to write the programs and everytime the variables are nameless and empty when i check the series window. It does the same with the xls data files. On the other hand, with another software like Jmulti it works, all the variables exist.
I attach some files here if someone wants to give it a try.
Thanks people.

The XLSX file has two sheets, the first of which has the "legend" and the second the data. The data sheet has no labels on it. If you use the data wizard, change to the data sheet, and give it names for the variables, it seems to work fine.

OPEN DATA "sims_data.xlsx"
CALENDAR(M) 1959:1
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS,SHEET="Data") 1959:01 1998:12 y1 y2 y3 y4 y5 y6 y7 y8 y9 y10 y11 $
y12 y13 y14 y15 y16 y17

I imagine you had to do something similar to get it to work with Jmulti.

The Matlab file is a single matrix (ytdata) with 130 rows and 7 columns. I assume the idea is to treat that as 7 separate series. Again, since there is no further information on the file, you'll have to provide the series names. Something like:

OPEN DATA "peersman.mat"
DATA(FORMAT=MATLAB,NOLABELS,SHEET="ytdata") 1 130 y1 y2 y3 y4 y5 y6 y7

[Nabtheberliner]

Dear Tom,
The message was sent before i finished it, very sorry. Here is the complete message
Thanks for your answer.

Concerning the sims_data.xlsx file,
indeed it's working,what is strange is that i did the same before getting to you with this matter by renaming the variables as in Jmulti ytdata_1 ytdata_2..... and didn't work??? Also with RATS in the wiard the legend sheet is the one containing the data, and the data sheet contains the legend, again???

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\sims_data.xlsx"
CALENDAR(Q) 1951:1
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS) 1951:01 2070:04 Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y10 Y11 Y12 Y13 Y14 Y15 $
Y16 Y17

You can notice,i don't have in the line: DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS), what you have SHEET="Data right after ORG=COLUMNS

Also i do the same with another xlsx file from the same book, named G7DATA.xlsx, attached below, and i get the problem even by renaming the variables? The view series shows 0 obs

.

Concerning the matlab file, i do exactly the same:

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\peersman.mat"
DATA(FORMAT=MATLAB,NOLABELS) 1951:01 1983:02 Y1 Y2 Y3 Y4 Y5 Y6 Y7

The view series shows nothing,and the output i get is:

Code: Select all

## IO9. Invalid input "ytdata" on line 0 while processing series Y1 entry 0
## IO30. There is no series Y2 on the file
## IO30. There is no series Y3 on the file
## IO30. There is no series Y4 on the file
## IO30. There is no series Y5 on the file
## IO30. There is no series Y6 on the file
## IO30. There is no series Y7 on the file


i don't know what to think??
Also with Jmulti there is no need to do anything, it loads everything and the variables are named properly and directly, ready to be used, surprising i have to say

If you have any idea, thanks a lot Tom

[TomDoan]

You can notice,i don't have in the line: DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS), what you have SHEET="Data right after ORG=COLUMNS

You need SHEET="Data" because the first sheet on the file is the "Legends" which has no data on it.

Concerning the matlab file, i do exactly the same:

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\peersman.mat"
DATA(FORMAT=MATLAB,NOLABELS) 1951:01 1983:02 Y1 Y2 Y3 Y4 Y5 Y6 Y7

You need the SHEET="YTDATA" option as shown in my earlier message. The wizard wasn't putting that in---this is a very rare case, most Matlab files with time series data have one matrix per variable rather than a rectangular matrix of unnamed data.

i don't know what to think??
Also with Jmulti there is no need to do anything, it loads everything and the variables are named properly and directly, ready to be used, surprising i have to say

Named what? There are no specific names on either one of those files.

Also i do the same with another xlsx file from the same book, named G7DATA.xlsx, attached below, and i get the problem even by renaming the variables? The view series shows 0 obs

With the G7data file, you have to skip the top two lines (which are descriptions of the series rather than usable series names) and put in the desired names. Using the wizard I get:

OPEN DATA "G7Data.xlsx"
CALENDAR(M) 1960:3
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS,TOP=3) 1960:03 1975:02 cangrowth canspread fragrowth fraspread gergrowth $
gerspread itagrowth itaspread jpngrowth jpnspread ukgrowth ukspread usgrowth usspread

which works fine.

[Nabtheberliner]

Hello Tom,
OK i got it, what i found is with the xlsx files the option SHEET="...", in the cases that you can read below,is not that much needed and it works,
Very strange, the wizard with the sims_data file shows me the legend data and when i open the preview the variables are there. When i swicth to the data, the preview shows the legend and doesn't contain the variables but just the legend, it's reversed but once i know it, it works.

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\sims_data.xlsx"
CALENDAR(M) 1951:1
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS) 1951:01 1990:12 Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y10 Y11 Y12 Y13 Y14 Y15 $
Y16 Y17

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\G7Data.xlsx"
CALENDAR(M) 1960:3
DATA(FORMAT=XLSX,NOLABELS,ORG=COLUMNS,TOP=3) 1960:03 1975:02 CAN_Y CANSPREAD FRA_Y FRASPREAD GER_Y GERSPREAD $
ITA_Y ITASPREAD JPN_Y JPNSPREAD UK_Y UKSPREAD US_Y USSPREAD NO


Concerning the matlab files it is needed otherwise indeed it doesn't work, but i load the file first with the wizars then RATS declare there is no variables, next i write the option SHEET="...", and it works

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\peersman.mat"
CALENDAR(Q) 1960:1
DATA(FORMAT=MATLAB,NOLABELS,SHEET="YTDATA") 1960:01 1992:02 S1 S2 S3 S4 S5 S6 S7

Code: Select all

OPEN DATA "C:\Users\naceur\Desktop\econometrics time series\MATLAB\MATLAB\portfolio_data.mat"
DATA(FORMAT=MATLAB,NOLABELS,SHEET="YTDATA") 1980:01 2005:02 R1 R2 R3 R4 R5 R6 R7


With JMulti, i load the dataset and the window shows

time/index ytdata_1 ytdata_2 ytdata_3 .................
1960 Q1 ......... ......... ...........
1960 Q2 ......... ......... ...........
1960 Q3 ......... ......... ...........
.
.
.
.
.

THEN I confirm the loading and it works as usual with the variables ytdata_1 ydata_2.......


Thanks a lot Tom

[TomDoan]

I would like to point out that neither the DATA(....) / y1 y2 ... y17 done with RATS nor the ytdata_1, ... that you are getting with JMulti are really giving you data that are ready to go. That's giving you data that are ready for major errors. Give the series real names (as is done in my code for the G7data) and before going on make sure that you have those labels correct. If I had a dollar for each Gauss or Matlab program that I've seen that misidentified data columns, I'd be rich.

[Nabtheberliner]

This is a precious advice!!

[quote][/quote]That's giving you data that are ready for major errors

Could you just precise a bit more what do you mean?
Do you mean, the expressions(letters, number) don't mean anything and by naming them properly we know what we are dealing with?
Thanks Tom

[TomDoan]

This is a precious advice!!

That's giving you data that are ready for major errors

Could you just precise a bit more what do you mean?
Do you mean, the expressions(letters, number) don't mean anything and by naming them properly we know what we are dealing with?
Thanks Tom

Unless Y1 to Y17 are parallel data for 17 countries or 17 firms, there is a very high chance that you will misidentify a data series. I've seen entire papers written with empirical work that assumed that column 5 in their data file was something completely different than what it actually was.

[Nabtheberliner]

Thanks again for the advise Tom, that's clear, i'll be cautious.

[Nabtheberliner]

Hi Tom,
If the data file that you want to open with RATS has no specific SHEET option when you open it with the wiard, like the one i attach here, what do you do in this case? where do you get the information?

[Nabtheberliner]

Sorry, i didn't finish, maybe because they have only one sheet?

[TomDoan]

Sorry, i didn't finish, maybe because they have only one sheet?

Right. There's only the content of the one file. That one is much better than the other files that you've posted because it has well-formatted column labels. The only thing missing are the dates---if you want those, you'll have to assign the start date and calendar scheme yourself.

[Nabtheberliner]

ok i see and it makes completly sens with your remark from yesterday. The thing is that a lot of data files are uncomplete in that sens, badly formatted, one more reason to be cautious.
Thanks Tom

[TomDoan]

ok i see and it makes completly sens with your remark from yesterday. The thing is that a lot of data files are uncomplete in that sens, badly formatted, one more reason to be cautious.
Thanks Tom

Unfortunately, the Martin-Hurn-Harris book is using an eclectic mix of original source data files and in some cases you can't actually tell what's on a file without looking at the Matlab or Gauss source code.

[Nabtheberliner]

Indeed the datasets are mixed up, i try to get the data for the example 13.19 US macro p.485 from what they provide as ressource but when i open it in RATS wizard, i have no clue about what i'm dealing with, so the only way is getting the matlab or gauss source code?
If i wanna delete some columns for instance column 3 and 5, assuming i have all together 8 columns?