hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@maprtech.com>
Subject Re: Cluster wide atomic operations
Date Sat, 27 Oct 2012 05:22:40 GMT
This is better asked on the Zookeeper lists.

The first answer is that global atomic operations are a generally bad idea.

The second answer is that if you an batch these operations up then you can
cut the evilness of global atomicity by a substantial factor.

Are you sure you need a global counter?

On Fri, Oct 26, 2012 at 11:07 PM, David Parks <davidparks21@yahoo.com>wrote:

> How can we manage cluster-wide atomic operations? Such as maintaining an
> auto-increment counter.
> Does Hadoop provide native support for these kinds of operations?
> An in case ultimate answer involves zookeeper, I'd love to work out doing
> this in AWS/EMR.

View raw message