hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Passing messages
Date Mon, 20 Dec 2010 03:24:24 GMT
On 12/18/2010 11:04 AM, Martin Becker wrote:
> Hello everbody,
> I am wondering if there is a feature allowing (in my case) reduce
> tasks to communicate. For example by some volatile variables at some
> centralized point. Or maybe just notify other running or to-be-running
> reduce tasks of a completed reduce task featuring some arguments.
> In my case, I have reduce tasks doing computations that will
> output/produce certain quality threshold. Other reduce tasks can/could
> estimate, if they ever get above those thresholds. If not they could
> just cease running.
> Thanks in advance,
> Martin
> PS: If such functions are not yet part of the API, I would like to
> know if there are good reasons for it and if not, propose to introduce
> such functionality.

You could use Zookeeper to do this.


View raw message