incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Tue, 04 Oct 2011 14:32:34 GMT

    [ https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120159#comment-13120159
] 

Thomas Jungblut commented on HAMA-431:
--------------------------------------

Thanks for the tutorial in the YARN site module. ;) Although it is not complete and sometimes
there are variables which were never declared it is really helpful.
I just added the container allocation and start of the sync server as a daemon inside of the
application master.
Then I added the BSPTaskLauncher which will then spawn the BSP tasks. 

I decided to not follow the statemachine handling like MapReduce, because I think this event
handling is worse than GOTOs are. It is not transparent (maybe not only to me?) from which
point the event handler is handling the events and stuff.
I don't think we need this actor model now and we should have a simple first snapshot which
works and is easy to develop with. 
BSP does not have this task execution like MapReduce and we don't need the capability to schedule
Tasks during the computation (excluding failure).

TODO:
- the real launching of the tasks within the BSPTaskLauncher
- task/job/overall cleanup
- the client integration
- a lot of testcases...
                
> MapReduce NG integration
> ------------------------
>
>                 Key: HAMA-431
>                 URL: https://issues.apache.org/jira/browse/HAMA-431
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: job_state.dot, task_phase.dot, task_state.dot
>
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
platform.
> Can be currently found in the 0.23 branch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message