Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <022201cdb3f0$2d5fd2f0$881f78d0$@yahoo.com>
References: <022201cdb3f0$2d5fd2f0$881f78d0$@yahoo.com>
From: Ted Dunning <tdunning@maprtech.com>
Date: Sat, 27 Oct 2012 01:22:40 -0400
Message-ID: 
 <CAND0qzsj7Gc+DXRHDrPkz3EUngkDTjcVH57KB5V1Z+bdOk7iWA@mail.gmail.com>
Subject: Re: Cluster wide atomic operations
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=14dae9399da74644b704cd03a108

--14dae9399da74644b704cd03a108
Content-Type: text/plain; charset=ISO-8859-1

This is better asked on the Zookeeper lists.

The first answer is that global atomic operations are a generally bad idea.

The second answer is that if you an batch these operations up then you can
cut the evilness of global atomicity by a substantial factor.

Are you sure you need a global counter?

On Fri, Oct 26, 2012 at 11:07 PM, David Parks <davidparks21@yahoo.com>wrote:

> How can we manage cluster-wide atomic operations? Such as maintaining an
> auto-increment counter.
>
> Does Hadoop provide native support for these kinds of operations?
>
> An in case ultimate answer involves zookeeper, I'd love to work out doing
> this in AWS/EMR.
>
>

--14dae9399da74644b704cd03a108
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

This is better asked on the Zookeeper lists.<div><br></div><div>The first a=
nswer is that global atomic operations are a generally bad idea.</div><div>=
<br></div><div>The second answer is that if you an batch these operations u=
p then you can cut the evilness of global atomicity by a substantial factor=
.</div>

<div><br></div><div>Are you sure you need a global counter?<br><br><div cla=
ss=3D"gmail_quote">On Fri, Oct 26, 2012 at 11:07 PM, David Parks <span dir=
=3D"ltr">&lt;<a href=3D"mailto:davidparks21@yahoo.com" target=3D"_blank">da=
vidparks21@yahoo.com</a>&gt;</span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">How can we manage cluster-wide atomic operat=
ions? Such as maintaining an<br>
auto-increment counter.<br>
<br>
Does Hadoop provide native support for these kinds of operations?<br>
<br>
An in case ultimate answer involves zookeeper, I&#39;d love to work out doi=
ng<br>
this in AWS/EMR.<br>
<br>
</blockquote></div><br></div>

--14dae9399da74644b704cd03a108--