incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Task distribution issue
Date Thu, 16 Dec 2010 07:55:13 GMT
Hi,

According to my understanding, it looks like a refactoring issue to
reduce the complexity and the memory footprint of master server by
removing back-calls of TaskInProgress and map structures. Right?

I think, that's a nice idea. But we didn't implement the task killing,
re-attempting functions yet. So, I would propose to schedule it for
0.2.1 release.

-Edward

On Thu, Dec 16, 2010 at 12:55 PM, Lin Chia-Hung <clin4j@googlemail.com> wrote:
> When working on BSPPeer fault tolerance
> (https://issues.apache.org/jira/browse/HAMA-199), I came across
> another issue - the task distribution currently employed in HAMA is
> done by GroomServer requesting tasks from BSPMaster, similar to one
> used in Hadoop mapreduce. This strategy has discussion at
> https://issues.apache.org/jira/browse/MAPREDUCE-278 regarding to e.g
> memory footprint, race condition, etc. Although this issue do not
> directly relate to the BSPPeer fault tolerance, the strategy
> (GroomServer requests tasks from BSPMaster) employed may have
> potential issue e.g. task can not be reschedule to the node expected.
>
> So I would like to know if there is any chance the HAMA task
> distribution mechanism may work toward this direction (proactive task
> assignment)?
>
> Thanks
>



-- 
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message