incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: Task re-attempt
Date Thu, 02 Feb 2012 07:51:30 GMT
Hi Edward,

I would like to get into this fault-tolerance thing ASAP, we have to
include this in our next release. This is the argument to not include hama
in production environments.
In my opinion, yes we need these Attempts. Due to various reasons:
- input split is bound to a specific index, related to the sorting of the
task ids
- theres a mapping in zookeeper for host:port->taskid

I want to tell you about the examples which use the master-client
architecture, which relies on the fact that the task's are sorted ascending.
If the mastertask fails, a reattempt won't break the ordering. Only the
host:port mapping must be updated in the zk and the other tasks have to
flush the caches and remap the znodes.
If you add a new task, you'll get a lot more pain than you actually want ;)

Attemps are fine, or is there a specific problem you want to avoid?

2012/2/2 Edward J. Yoon <edwardyoon@apache.org>

> Few Task-related classes e.g., TaskAttemptID .., etc. are copied from
> Hadoop MapReduce.
>
> Do you think we need to implement Task re-attempt mechanism?
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Thomas Jungblut
Berlin <thomas.jungblut@gmail.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message