incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Tue, 13 Sep 2011 08:56:09 GMT


Thomas Jungblut commented on HAMA-431:

With polling, it seems that chances are the polling would not reach the agreement (there could
always have 1 process missing) in an unfortunate timing case. Also, as the processes increase
probably it would increase the loading for master to deal with polling tasks. 

This is correct, but depends highly on the polling intervall. As far as I understood each
BSPJob gets its own Application Master. So there is no "Master-Machine" anymore like our BSPMaster
or Jobtracker. 

We have two options:
- fix the barrier sync with zookeeper and use it in the AM and Peers
- do the polling in Application Master

In addition, my understanding is the integration with MRV2 would be just an additional support
so that MR job/ application can be submitted without rewriting to use hama for computation.

That is right as well. I think we should make this a configuration based decision whether
YARN (or the URL) has been set or not.

> MapReduce NG integration
> ------------------------
>                 Key: HAMA-431
>                 URL:
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
> Can be currently found in the 0.23 branch.

This message is automatically generated by JIRA.
For more information on JIRA, see:


View raw message