Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 63646 invoked from network); 29 Jan 2008 06:41:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 29 Jan 2008 06:41:51 -0000 Received: (qmail 88847 invoked by uid 500); 29 Jan 2008 06:41:43 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 88724 invoked by uid 500); 29 Jan 2008 06:41:42 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 88715 invoked by uid 500); 29 Jan 2008 06:41:42 -0000 Delivered-To: apmail-lucene-hadoop-commits@lucene.apache.org Received: (qmail 88712 invoked by uid 99); 29 Jan 2008 06:41:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Jan 2008 22:41:42 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.130] (HELO eos.apache.org) (140.211.11.130) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Jan 2008 06:41:24 +0000 Received: from eos.apache.org (localhost [127.0.0.1]) by eos.apache.org (Postfix) with ESMTP id D1CEAD2DB for ; Tue, 29 Jan 2008 06:41:30 +0000 (GMT) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Apache Wiki To: hadoop-commits@lucene.apache.org Date: Tue, 29 Jan 2008 06:41:30 -0000 Message-ID: <20080129064130.6191.45382@eos.apache.org> Subject: [Hadoop Wiki] Trivial Update of "Matrix" by udanax X-Virus-Checked: Checked by ClamAV on apache.org Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification. The following page has been changed by udanax: http://wiki.apache.org/hadoop/Matrix ------------------------------------------------------------------------------ == Hbase Matrix Package for Map/Reduce-based Parallel Matrix Computations == - The matrix package will be useful for some of the Large-Scale Numeric Analysis and Data Mining which need the computation system of the Inverse Matrix for Data Mining related area (e.g. linear regression, PCA, SVM, ..., etc). + The matrix package will be useful for Large-Scale Numeric Analysis and Data Mining which need the computation system of the Inverse Matrix (e.g. linear regression, PCA, SVM, ..., etc). - Generally, The current shared-memory based parallel matrix solution provides a scalable and high performance matrix operations, however, matrix resources can't be scalable. But, Using Hbase's Row,Column(Qualifier) two dimensional space, we are able to store large sparse matrix. Also, The Auto-partitioned sparsity sub-structure will be efficiently managed and serviced by Hbase. Row or Column operations can be done in linear time and algorithms such as structured Gaussian elimination or iterative methods run in O(~-the number of non-zero elements in the matrix-~ / ~-number of mappers (processors/cores)-~) time on Map/Reduce. + Generally, The current shared-memory based parallel matrix solution provides a scalable and high performance matrix operations, but, matrix resources can't be scalable. But, Using Hbase's 2- dimensional Row and Column(Qualifier) space, we are able to store large sparse matrix. Also, auto-partitioned sparsity sub-structure will be efficiently managed and serviced by Hbase. Row or Column operations can be done in linear time and algorithms such as structured Gaussian elimination or iterative methods run in O(~-the number of non-zero elements in the matrix-~ / ~-number of mappers (processors/cores)-~) time on Map/Reduce. === Initial Contributors === @@ -36, +36 @@ * Cholesky Decomposition === Getting Start === + Download the matrix package: + {{{ + bash# wget http://wiki.apache.org/hadoop-data/attachments/Matrix/attachments/matrix-test_v0.0.1.tar + bash# ant package + }}} + After setup the configuration files of Hadoop + Hbase, type in the following: {{{ bash# ./bin/hadoop jar ./lib/hadoop-0.16.0-dev-hbase.jar org.apache.hadoop.hbase.matrix.ExampleDriver }}} ---- == Future Plans == - * it needs own Input/Output formatter and splitter. + * It needs own Input/Output formatter and splitter. - + * Make the Decompositions and Factorizations Map/Reduce classes. ---- == References ==