Return-Path: Delivered-To: apmail-zookeeper-user-archive@www.apache.org Received: (qmail 77414 invoked from network); 8 Mar 2011 18:14:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 8 Mar 2011 18:14:24 -0000 Received: (qmail 91384 invoked by uid 500); 8 Mar 2011 18:14:23 -0000 Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org Received: (qmail 91351 invoked by uid 500); 8 Mar 2011 18:14:23 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 91343 invoked by uid 99); 8 Mar 2011 18:14:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Mar 2011 18:14:23 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.97.132.74] (HELO homiemail-a37.g.dreamhost.com) (208.97.132.74) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Mar 2011 18:14:16 +0000 Received: from homiemail-a37.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a37.g.dreamhost.com (Postfix) with ESMTP id 0328920806D for ; Tue, 8 Mar 2011 10:13:55 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=darose.net; h=message-id:date :from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=darose.net; b= mCxG37PX1ii+RQDhGtntLOYQsrVWnBONU3yEAaUqb2TBEewXII0taEQTxhCT5UGK M6WfAQ5Wcivhpaxv4R+ZBPq2C7hG8tPekOIYWiuZ8gJA9T8orRPMtnABQQcacnsp oX+dCnD6My531YLM9693beZezhhwNCMGEIEgYQ9AUDo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=darose.net; h=message-id :date:from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; s=darose.net; bh=ifsqei 8/Fla8RkjMzTj0hXH2kBs=; b=Ppu2rf6n9S4wmnBOolkL6pLeT99X5hQtjOmavz Pss+7bBx9/CRXDQzLqeKjCMxINRAmD+9sKsePvk5S1ZAhDg3dYDDMd0fIKaqASti R5nGj3UUz+s0rKpXb/7JSPq6KjhIz4HAvw+o1k31EsCEU64RDqEAW9VFg1lfxer4 ji2Ck= Received: from darsys12 (cpe-66-108-142-190.nyc.res.rr.com [66.108.142.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: darose@darose.net) by homiemail-a37.g.dreamhost.com (Postfix) with ESMTPSA id CD75A20806A for ; Tue, 8 Mar 2011 10:13:54 -0800 (PST) Received: from [10.1.0.2] by darsys12 with esmtp (Exim 4.74) (envelope-from ) id 1Px1Pt-0006dH-LH for user@zookeeper.apache.org; Tue, 08 Mar 2011 13:13:53 -0500 Message-ID: <4D7671E1.6070104@darose.net> Date: Tue, 08 Mar 2011 13:13:53 -0500 From: David Rosenstrauch User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110304 Lanikai/3.1.9 MIME-Version: 1.0 To: user@zookeeper.apache.org Subject: Re: Zookeeper for generating sequential IDs References: <4D7670D5.4050301@darose.net> In-Reply-To: <4D7670D5.4050301@darose.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org On 03/08/2011 01:09 PM, David Rosenstrauch wrote: > On 02/25/2011 04:50 AM, Ertio Lew wrote: >> Hi all, >> >> I am involved in a project where we're building a social application >> using Cassandra DB and Java. I am looking for a solution to generate >> unique sequential IDs for the content on the application. I have been >> suggested by some people to have a look to Zookeeper for this. I >> would highly appreciate if anyone can suggest if zookeeper is suitable >> for this purpose and any good resources to gain information about >> zookeeper. >> >> Since the application is based on a eventually consistent distributed >> platform using Cassandra, we have felt a need to look over to other >> solutions instead of building our own using our DB. >> >> Any kind of comments, suggestions are highly welcomed! :) >> >> Regards >> Ertio Lew. > > I ran into a similar id-generation issue, and wrote a library for it. > (Details described in this msg: > http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/201008.mbox/%3C4C5B7656.4020200@darose.net%3E > .) > > Been planning to release it to the community, but haven't gotten around > to it yet. > > Not sure my solution is exactly what you're looking for though. > > HTH, > > DR BTW, that email is old. We now have had this running quite reliably in production for several months now. It's being used by M/R jobs running 100 simultaneous reducers, each accessing the ID generator, and assigning nearly 1 million ID's per job in total. DR