hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Trivial Update of "Hbase/Matrix" by udanax
Date Mon, 22 Oct 2007 13:57:25 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/lucene-hadoop/Hbase/Matrix

------------------------------------------------------------------------------
  [[TableOfContents(4)]]
  ----
- == HbaseLINA, a framework for large scale sparse linear algebra ==
+ == HbaseLINA, a Framework for Large-scale Sparse Linear Algebra ==
  
  Using [:Hbase:Hbase]'s Row,Column(Qualifier) two dimensional space, we are able to store
large sparse matrix.
  [[BR]]The Auto-partitioned sparsity sub-structure will be efficiently managed, serviced
by [:Hbase:Hbase].
  
- Also using iterative algorithms like conjugate gradient method on a parallel processing
platform like MapReduce, 
- [[BR]]we should be able to implement '''High Performance''' and '''World's Largest''' Matrix
Computations.
+ Row or Column operations can be done in linear time and algorithms such as structured Gaussian
elimination
+ [[BR]]or iterative methods run in '''O(~-the number of non-zero elements in the matrix-~)'''
time, [:Hbase:Hbase] Providing Both Vertical and Horizontal access.
  
- The HbaseLINA application language will be added as sub-shell in [:Hbase/HbaseShell:Hbase
Shell].
+ Therefore, using iterative algorithms on a parallel processing platform like MapReduce,

+ [[BR]]we should be able to implement ''High Performance'' and ''World's Largest'' Matrix
Computations.
  
  === Applications ===
  
@@ -31, +32 @@

  
  == Storing and manipulating numeric, sparse matrices on Hbase ==
  
- A Sparse matrix is created by following the creation command with the keyword "Matrix".
+ === Scheduling Algorithm ===
  
- NOTE :
+ The scheduling algorithm is designed for driving the parallel execution of the factorization
on on a MapReduce model.
  
+ === Hbase Shell Extension ===
- {{{
- difficulties in tuning sparse matrix. 
-  - indirect and irregular, the inefficient data access 
  
+ The HbaseLINA application language will be added as sub-shell in [:Hbase/HbaseShell:Hbase
Shell].
- reordering for matrix partitioning (to create dense structure.) 
-  - graph partitionning algorithms ( use hypergraph for non symmetric matrices ) 
-  - TDF ordering 
- }}}
- 
- 
- 
- === Hbase Shell Language Extension ===
- 
  {{{
  Hbase > lina;
  
  Hbase LINA version 0.0.1
  Type 'help;' for help.
  
- Hbase.LINA > M = Matrix('m_table','cf_1'); //Set up the matrix M from mapped matrix in
hbase.
+ Hbase.LINA > M = Matrix('m_storage'); //Set up the matrix M from mapped matrix in hbase.
  Hbase.LINA > R = M.SVDdecomposition();
  Hbase.LINA > U = R.getU();
- Hbase.LINA > Save U into Matrix('table_name','Othonormal_eigenvector_U');
+ Hbase.LINA > Save U into Matrix('Othonormal_eigenvector_U');
  ...
  
  Hbase.LINA > exit;
@@ -68, +59 @@

  == References ==
  
   * [http://bebop.cs.berkeley.edu/oski/ OSKI], optimized sparse kernel interface (OSKI) library
+   * [http://icl.cs.utk.edu/iclprojects/pages/files/sans/yelick-bebop.pdf Automatic Performance
Tuning of Sparse Matrix Kernels], August 7, 2002
   * Parallel Conjugate Gradients Assignment, a parallel implementation of the conjugate gradient
algorithm
   * ScaLAPACK, a library of high-performance linear algebra routines for distributed-memory
message-passing MIMD computers 
   * Scheduling algorithms for parallel Gaussian elimination withcommunication costs, Amoura,
A.K.; Bampis, E.; Konig, J.-C.

Mime
View raw message