Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=darose.net; h=message-id:date
	:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=darose.net; b=
	MLsUfVf8m3EnbctHT06rkhGgBPCIjJbWTvYfipzos4pY6fepUoSS5FU0ygiOvTv6
	WMdy0YPe2ZjwOhTAphbtyJ6s6XUm/1xUoCkZFeV/YftjGrI4fwLn6VtPYvh+WFtb
	B9M8bCcMobBWenCMiEuzV4dhOl7DdAgCod8fRNoQ8D8=
Message-ID: <4CF7D594.7000206@darose.net>
Date: Thu, 02 Dec 2010 12:21:24 -0500
From: David Rosenstrauch <darose@darose.net>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
 rv:1.9.2.12) Gecko/20101028 Lanikai/3.1.6
MIME-Version: 1.0
To: user@zookeeper.apache.org
Subject: Re: counter with zookeeper
References: <4CF7AD46.9040309@tis.bz.it>
 <AANLkTinsM+BPHOSnXJok8Qr-hsW0PQ0Ep1P0hMQZBJaM@mail.gmail.com>
 <4CF7BDF5.60905@tis.bz.it>
 <AANLkTikjuCSHAt9o+jjfRC2ojdj08Mbx=7p0pGnsvp4f@mail.gmail.com>
 <4CF7C163.7020300@darose.net> <4CF7C243.3060305@tis.bz.it>
In-Reply-To: <4CF7C243.3060305@tis.bz.it>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

I don't, for several reasons:

a) We request ID's frequently enough that old ones do eventually get 
used up.

b) Even with a largely fragmented ID space, the largest we've seen the 
bytes contents of that ZK node get up to is 5-6KBytes.  So no real 
worries about either storage space or network I/O when reading/writing 
to/from the ZK node.

DR

On 12/02/2010 10:58 AM, Claudio Martella wrote:
> I like Ted's idea too.
>
> David, how do you handle re-compaction of your fragmented ID space?
>
>
> On 12/2/10 4:55 PM, David Rosenstrauch wrote:
>> On 12/02/2010 10:47 AM, Ted Dunning wrote:
>>> I would recommend that you increment the counter by 100 or 1000 and then
>>> increment a local counter over the implied range.  This will drive the
>>> amortized ZK overhead down to tens of microseconds which should be
>>> good for
>>> almost any application. Your final ids will still be almost entirely
>>> contiguous.  You could implement a fancier counter in ZK that remembers
>>> returned chunks for re-use to get perfect contiguity if you really
>>> wanted
>>> that.
>>
>> This is what our library does.  You request chunks of, say, 1000 ID's,
>> and then push back any remaining unused ID's in the chunk you took.
>>
>> DR