incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Mon, 03 Oct 2011 17:49:34 GMT

    [ https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119434#comment-13119434
] 

Thomas Jungblut commented on HAMA-431:
--------------------------------------

{quote}
Don't import or copy over the ContainerLauncher. All you need is the code in there to start
and stop containers.
{quote}

Oh okay. I'll remove them later. Can you provide a tiny code example what is needed to launch
a container and how it should look like in our cases?

{quote}
Regarding the state machines, we will need some kind of representation for the barrier sync
in the job too.
{quote}

I would not track this via the state machine. But it can be possible if we can integrate the
sync daemon into the application master.
So the application master will take care of the sync. Aren't the RPC services blocking each
other then? I tested an integration with our Groom and BSPMaster, and it totally failed, so
I had to put this into another process.

{quote}
Seems like Sync daemon can be started by the ApplicationMaster itself.
{quote}

Yes, there is still the question how to get the host:port of this daemon after it has been
launched. Is there a kind of communication between the starter and the container?

{quote}
Still not sure about the checkpointer.
{quote}

Me neither, so we can leave this point open and revise the checkpointing mechanism later.
I don't want to be inconsistent to the current features, but I think each task has to make
its own checkpointing. 

{quote}
I'll be hanging around at #hama channel at freenode, we can sync up w.r.t implementation details.
(My timezone is IST)
{quote}

I was a few days off so I wasn't there, you probably noticed, I'm sorry.

                
> MapReduce NG integration
> ------------------------
>
>                 Key: HAMA-431
>                 URL: https://issues.apache.org/jira/browse/HAMA-431
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: job_state.dot, task_phase.dot, task_state.dot
>
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
platform.
> Can be currently found in the 0.23 branch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message