We're trying to determine B where BB' is a factorization of sigma. A contemporaneous zero restriction will take the form B(i,j)=0. A row of the R matrix will have N**2 elements; in this case, it will be all zeros except for a one in the slot for B(i,j) in the vec'ed form of B. RATS vec's by columns, so, for instance, if this is a 3x3 matrix, B(1,3) will be the 7th slot in the vec. If you want to restrict shock 1 to load equally on the 2nd and 3rd variables (B(2,1)=B(3,1)), the "R" row for that would be (0,1,-1,0,...,0).
The long run responses are inv(phi(1))B (done as a standard matrix multiply). The i,j element of that will be row i of inv(phi(1)) dot column j of B, so the row in R to make that 0 is zeros everywhere but the slots corresponding to column j in B; and those slots will have the values of row i. Again, with the 3 x 3 matrix, suppose the long run restriction is (3,2)=0. If we define A=inv(phi(1)) (remember that this is a known matrix - it's B that's unknown), the row in R for this restriction is (0,0,0,a(3,1),a(3,2),a(3,3),0,0,0). If we want the long-run restriction to be (3,1)=0, the row in R is (a(3,1),a(3,2),a(3,3),0,0,0,0,0,0).
So far as I know, the first paper to combine short and long run restrictions was Gali, "How Well Does the IS-LM Model Fit Postwar U.S. Data", QJE 1992, vol 107, no. 2, pp 709-738. That however has the complication that it also includes restrictions on the structure of the shocks (inv(B)) so it doesn't have a simple reparameterization.