incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Wed, 12 Oct 2011 08:43:12 GMT

    [ https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125666#comment-13125666
] 

Thomas Jungblut commented on HAMA-431:
--------------------------------------

Okay I fixed the last issues.

I would now open some subtasks. For example to cleanup the TODOs or test with a distributed
YARN (currently working with pseudo distribution). 
Especially profiling the ApplicationMaster would be a task too. Its container will be killed
if less than 2gb were allocated, I don't think that this is using that much memory (maybe
its a misconfiguration).
Then we should take a look at the module splitting. Currently the server package is just on
top of API/Core, which is in trunk solely core.
Although I think it is not a bad design, we have several new classes for the YARN stuff e.G.
the YARNBSPJob. I don't know if we really should integrate YARN into our BSPJob. 
Then we have to catch up the sources to the current trunk.

BTW you can build the server module with mvn install package and use the shaded jar to run
on yarn with: yarn/bin/yarn jar <JAR>.jar org.apache.hama.bsp.YarnSerializePrinting

It is currently not working as expected, I'm still facing some conf and classpath issues.
But I hope I'll finish them today.
                
> MapReduce NG integration
> ------------------------
>
>                 Key: HAMA-431
>                 URL: https://issues.apache.org/jira/browse/HAMA-431
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: job_state.dot, task_phase.dot, task_state.dot
>
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
platform.
> Can be currently found in the 0.23 branch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message