Mailing-List: contact zookeeper-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: zookeeper-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <4CBE1346.2080307@yahoo-inc.com>
Date: Tue, 19 Oct 2010 14:53:10 -0700
From: Benjamin Reed <breed@yahoo-inc.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.9.2.9) Gecko/20100922 Thunderbird/3.1.4
MIME-Version: 1.0
To: zookeeper-user@hadoop.apache.org
Subject: Re: zxid integer overflow
References: 
 <A0B699C30D50C84883E79752AEA74B881D74C6C761@NAMBX01.corp.adobe.com>
In-Reply-To: 
 <A0B699C30D50C84883E79752AEA74B881D74C6C761@NAMBX01.corp.adobe.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

  we should put in a test for that. it is certainly a plausible 
scenario. in theory it will just flow into the next epoch and everything 
will be fine, but we should try it and see.

ben

On 10/19/2010 11:33 AM, Sandy Pratt wrote:
> Just as a thought experiment, I was pondering the following:
>
> ZK stamps each change to its managed state with a zxid (http://hadoop.apache.org/zookeeper/docs/r3.2.1/zookeeperInternals.html).  That ID consists of a 64 bit number in which the upper 32 bits are the epoch, which changes when the leader does, and the bottom 32 bits are a counter, which is incremented by the leader with every change.  If 1000 changes are made to ZK state each second (which is 1/20th of the peak rate advertised), then the counter portion will roll over in 2^32 / (86400 * 1000) = 49 days.
>
> Now, assuming that my math is correct, is this an actual concern?  For example, if I'm using ZK to provide locking for a key value store that handles transactions at about that rate, am I setting myself up for failure?
>
> Thanks,
>
> Sandy