ewise for matrix and vector (and %cputime() request)

[randal_verbrugge]

Hi all,

I'm using some large matrices (42000 x 13), and trying to speed up my code using ewise rather than loops. (btw for optimization purposes, it would be helpful for %cputime() to output a real, rather than integer).

Instead of a loop like
do j = 1, 42000
* Shift lagged values first
do lag = 13, 2, -1
compute theta_star(j,lag) = theta_star(j,lag-1)
end do lag
end do j

I wanted to use ewise to shift data from one column to the next.
RATS does not return an error when I use:
do lag = 13, 2, -1
ewise theta_star(i,lag) = theta_star(i,lag-1)
end do lag

In fairness, the ewise instruction writeup does say that this should not work, since in each step, it is setting only one column. However, running this code, RATS churns away, seeming to be doing something. But In keeping with that writeup, all this time is spent filling the matrix with NA (though again, without an error message). The writeup does not provide details as to why this is problematic.

However, in accord with the prohibition stated in the writeup, ewise does express unhappiness when it comes to setting the first column of theta_star. The following was originally in the loop above:
do j = 1, 42000
* Shift lagged values first
...
* * Update theta_star (driftless random walk)
compute theta_star(j,1) = theta_star(j,2) + unit_std*rand_theta(j)
end do j

to replace this, RATS doesn't like any of the following:


ewise theta_star(j,1) = theta_star(j,2) + unit_std*rand_theta(j)

or

dec vect thetaCol1(42000)
ewise thetaCol1(i) = theta_star(i,2) + unit_std*rand_theta(i) ; *OK so far
ewise theta_star(i,1) = thetaCol1(i) ; *nope.

or

dec rect ThetaCol1b(42000,1)
ewise thetaCol1b(i,1) = theta_star(i,2) + unit_std*rand_theta(i) ; *nope.

or
dec rect RandThetaArray(42000,1)
do i=1,42000; compute randthetaarray(i,1) = rand_theta(i)
end do i
ewise thetaCol1b(i,1) = theta_star(i,2) + unit_std*randthetaarray(i,1); *nope.

I also notice that
%psubmat(A,startrow,startcol,B) and %psubvec(VA,position,VB) seem like they would be useful, but you have to go array to array, OR vector to vector, but not vector to column of an array.

This is what I found (from instruction writeup) that does work.
compute %do(i,1,42000,theta_star(i,1)=thetaCol1(i))

Note that compute %do seems much more flexible, e.g. it is also OK with mixing a constant with an expression from a different array:
compute %do(i,1,42000,rent_star(i,1)=R_agg(date_index) + theta_star(i,1))

Not sure how compute %do compares to ewise with respect to speed (or for that matter, compared to an explicit do loop).

Anyway, any further insight into how I can speed this shifting-of-columns up, or any further description of what ewise likes, and what it does not like, and the reasons why going from a vector to a column of an array is challenging, would be helpful!

many thanks,
R