Does SystemML support logical indexing?
For example if X is a numerical matrix with 2 columns and n rows (in my
case n ~ 35 million). I'd like to split the matrix rowwise according to
values of the first column. This is useful when I need to find
distributions of subgroups of population. In R I can do
Y = X[ X[ ,1] > 10, ]
OR
ind = which(X[ ,1] > 10)
Y = X[ind, ]
It seems neither syntex works in SystemML.
I noticed there's an aggregate() function for SystemML, but it supports
coded categorical variable.
Perhaps one way to do that is creating an indicator n by 1 matrix Z that
takes values 1 and 2 where 1 corresponds to X[, 1] <= 10 and 2 corresponds
to X[,1] > 10. Then aggregate() X[,2] with respect to Z.
It seems transform() with 'bin' option is one obvious way to create such a
Z, however the 'bin' method only supports 'equiwidth' currently.
Is looping through X[,1] the best option? Maybe I missed some other
convenient functions.
Any suggestions are greatly appreciated!
Best,
Ethan
