systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arijit chakraborty <ak...@hotmail.com>
Subject Re: Combining Multiple Matrix
Date Wed, 03 May 2017 10:14:52 GMT
Thank you Fred! We are actually doing what you suggested. We were concerned about the large
"null" matrix we are generating at the beginning of the process. But if large zero matrix
is not a performance issue, then I guess we are on the right track.


Thanks a lot!

Arijit




________________________________
From: Frederick R Reiss <frreiss@us.ibm.com>
Sent: Wednesday, May 3, 2017 5:14 AM
To: dev@systemml.incubator.apache.org
Subject: Re: Combining Multiple Matrix


Hi Arijit,

Have you considered using parfor and left indexing? Here's an example from the docs that generates
a large matrix out of smaller chunks:

ms = matrix(0, rows=2, cols=3*10)
parfor (v in 1:10, check=0) {
   mv = matrix(v, rows=2, cols=3)
   ms[,(v-1)*3+1:v*3] = mv
}

SystemML has a special data structure to represent a matrix block that is all zeros, so creating
a very large matrix of zeros (first line in the above listing) is an inexpensive operation.

Fred

[Inactive hide details for arijit chakraborty ---05/02/2017 09:48:13 AM---Hi, Our process
is generating multiple matrix, of same]arijit chakraborty ---05/02/2017 09:48:13 AM---Hi,
Our process is generating multiple matrix, of same size, but number of matrix is random. Finally

From: arijit chakraborty <akc14@hotmail.com>
To: "dev@systemml.incubator.apache.org" <dev@systemml.incubator.apache.org>
Date: 05/02/2017 09:48 AM
Subject: Combining Multiple Matrix

________________________________



Hi,


Our process is generating multiple matrix, of same size, but number of matrix is random. Finally,
we combine all the smaller matrix to get a final large matrix.


The way we are creating the large matrix is, we are creating a blank matrix and then update
the matrix once each smaller matrix is created. This is inefficient because we are initially
creating a large matrix, which sometimes take more than required space.


So is there any method where we can combined all the final smaller matrix together into a
larger matrix? For rbind and cbind, we have to use loops to merge the matrix.


Thank you!

Arijit




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message