incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Wed, 07 Sep 2011 11:25:10 GMT

    [ https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098854#comment-13098854
] 

Thomas Jungblut commented on HAMA-431:
--------------------------------------

Checked in first version of BSPAppMaster and the job stuff (event/implementation etc). 

We need to rethink our states and transitions, but I think we can barely cut them down to
Setup->Compute->Cleanup.
A more sophisticated way would be to split the compute state into the supersteps. So each
supersteps gets handled by a statetransition. But that could introduce a lot of overhead,
but it seems to be cleaner than the handling of zookeeper. E.G. we can wait until every task
has reached this transition with a cyclic barrier. Just like in the LocalBSPRunner.

And we need to write the event dispatchers for them. 
My next goal would be to rewrite the BSPJobImpl class for our needs e.G. remove the map and
reduce tasks counter and extract an interface.

> MapReduce NG integration
> ------------------------
>
>                 Key: HAMA-431
>                 URL: https://issues.apache.org/jira/browse/HAMA-431
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
platform.
> Can be currently found in the 0.23 branch.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message