hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-2878) Hama code contribution
Date Fri, 23 May 2008 08:53:55 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Edward J. Yoon resolved HADOOP-2878.

    Resolution: Won't Fix

It moved to incubator. 

> Hama code contribution
> ----------------------
>                 Key: HADOOP-2878
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2878
>             Project: Hadoop Core
>          Issue Type: New Feature
>         Environment: All environment
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>            Priority: Minor
>         Attachments: hama_v01.patch
> *Introduction*
> Hama will develop a high-performance and large-scale parallel matrix computational package
based on Hadoop Map/Reduce. It will be useful for a massively large-scale Numerical Analysis
and Data Mining, which need the intensive computation power of matrix inversion, e.g. linear
regression, PCA, SVM and etc. It will be also useful for many scientific applications, e.g.
physics computations, linear algebra, computational fluid dynamics, statistics, graphic rendering
and many more.
> Hama approach proposes the use of 3-dimensional Row and Column (Qualifier), Time space
and multi-dimensional Columnfamilies of Hbase (BigTable Clone), which is able to store large
sparse and various type of matrices (e.g. Triangular Matrix, 3D Matrix, and etc.). its auto-partitioned
sparsity sub-structure will be efficiently managed and serviced by Hbase. Row and Column operations
can be done in linear-time, where several algorithms, such as structured Gaussian elimination
or iterative methods, run in O(the number of non-zero elements in the matrix / number of mappers)
time on Hadoop Map/Reduce.
> So, it has a strong relationship with the hadoop project, and it would be great if the
"hama" can become a contrib project of the hadoop
> *Current Status*
> In its current state, the 'hama' is buggy and needs filling out, but generalized matrix
interface and basic linear algebra operations was implemented within a large prototype system.
In the future, We need new parallel algorithms based on Map/Reduce for performance of heavy
decompositions and factorizations. It also needs tools to compose an arbitrary matrix only
with certain data filtered from hbase array structure.
> It would be great if we can collaboration with the hadoop members.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message