incubator-hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Trivial Update of "Architecture" by udanax
Date Tue, 10 Mar 2009 08:56:14 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/hama/Architecture

------------------------------------------------------------------------------
  ----
  = Overview =
  
- This is intended to explain and illustrate the concept of Hama. There are three main parts:

+ This is intended to explain and illustrate the concept of Hama. There are two main parts:

+  
+  * How to store the matrices?
+  * How to perform matrix operations using !MapReduce?
  
  == Building Block ==
  
  [http://wiki.apache.org/hama-data/attachments/Architecture/attachments/block.png]
  
+ == Store Dense/Sparse Matrices ==
+ 
  To store the matrices, Hama use a [http://hadoop.apache.org/hbase/ Hbase] -- Matrices are
basically tables. They are ways of storing numbers and other things. Typical matrix has rows
and columns. Actually called a 2-way matrix because it has two dimensions. For example, you
might have respondents-by-attitudes. Of course, you might collect the same data on the same
people at 5 points in time. In that case, you either have 5 different 2-way matrices, or you
could think of it as a 3-way matrix, that is respondent-by-attitude-by-time.
  
  -- ''Just a thought, considering the depleted activity in HBase should we not explore ways
to avoid HBase ? --Prasen ''
  
- ----
- 
- = Dense Matrix =
- For dense matrix computations, The block-partitioned algorithms used to minimize data movement
and network cost. Dense Matrix and Blocked Dense Matrix are both stored in one table with
other metadata. But, Blocked dense matrix can't be synchronized by dense matrix update.
- 
- {{{
-   // Generate matrix with random elements
-   DenseMatrix a = DenseMatrix.random(conf, 1000, 1000);
-   DenseMatrix b = DenseMatrix.random(conf, 1000, 1000);
-   
-   // The type of the Matrix to be blocked, must be dense.
-   a.blocking(2);
-   b.blocking(2);
- 
-   DenseMatrix c = a.mult(b);
- }}}
- 
- For example, The matrix multiplication of the original arrays can be transformed into matrix
multiplication of blocks as describe below.
- 
- {{{
- C_block(1,1)=A_block(1,1)*B_block(1,1) + A_block(1,2)*B_block(2,1)
- 
- C                 A               B
- +-----+-----+     +-----+-----+   +-----+-----+
- | x x |     |     | --> | --> |   | | | |     |
- | x x |     |     | --> | --> |   | ↓ ↓ |     |
- +-----+-----+  =  +-----+-----+ * +-----+-----+
- |     |     |     |     |     |   | | | |     |
- |     |     |     |     |     |   | ↓ ↓ |     |
- +-----+-----+     +-----+-----+   +-----+-----+
- }}}
- 
-   -- To be statically sized blocks, What should we do? -- Edward
- 

Mime
View raw message