Return-Path: X-Original-To: apmail-zookeeper-user-archive@www.apache.org Delivered-To: apmail-zookeeper-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2B9A217783 for ; Wed, 4 Feb 2015 20:12:47 +0000 (UTC) Received: (qmail 71172 invoked by uid 500); 4 Feb 2015 20:12:47 -0000 Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org Received: (qmail 71129 invoked by uid 500); 4 Feb 2015 20:12:47 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 71107 invoked by uid 99); 4 Feb 2015 20:12:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Feb 2015 20:12:47 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tivv00@gmail.com designates 209.85.213.176 as permitted sender) Received: from [209.85.213.176] (HELO mail-ig0-f176.google.com) (209.85.213.176) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Feb 2015 20:12:21 +0000 Received: by mail-ig0-f176.google.com with SMTP id hl2so37434214igb.3 for ; Wed, 04 Feb 2015 12:10:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:from:date:message-id:subject:to:content-type; bh=Ydj/3gbIL15bEFZgmBoNt1ebQjzeWJmzJNpk6OPGXEU=; b=cDisaNf9YevZNzBTZeUfg2l7vr0K56XSdTifzb6nGUZRlYXkIeJ8jfc9OvycdXfyTR +xSr8nV2QtG2VCSwy3q4FLTC2Az0ejHO7vLqRWa3Bq/H0sUZdr3qs8DAMHIMgncOeyKy jFBZ03oTn6zAIfvahp4wu+gAph0AO3WQzutx5bKfOyRgTkKtmr13RTSEJ0B3+ScILBPa r3d+EHGOYwLGYw5TM00QRihFyWOd1pnzstx7WprW/yAlhwawyGzW+Savql8Ze+YxC0rX lsjbAdbD69X7hSeScyDKv3fJrMP5svmyu+hxYQnsRMGzB08fvr8Sr00mqlsqzknVgfqR e/nw== X-Received: by 10.50.83.10 with SMTP id m10mr26249907igy.23.1423080649863; Wed, 04 Feb 2015 12:10:49 -0800 (PST) MIME-Version: 1.0 From: Vitalii Tymchyshyn Date: Wed, 04 Feb 2015 20:10:49 +0000 Message-ID: Subject: Implementing sharding with zookeeper To: user@zookeeper.apache.org Content-Type: multipart/alternative; boundary=089e0122acca9f37ba050e48c83b X-Virus-Checked: Checked by ClamAV on apache.org --089e0122acca9f37ba050e48c83b Content-Type: text/plain; charset=UTF-8 Hello. I have a task that looks like a fit for Zookeeper and I thought I find a recipe in Curator, but I dont. So, It looks like I need my own recipe. The problem is: we have a stream of tasks coming in. All tasks for the same account must be done sequential. But for different accounts it's OK to do in parallel, so we want workers to distribute account space between them. The first step, of course, is to make an account hash to ensure load is distributed as smooth as possible. Next I could simply use a lot of locks, but this way I can't ensure fairness. Thats why I think I need a Leader who will distribute (or redistribute in case of node death) the hash space between the live nodes. What do you think? May be I am missing something and there is existing recipe for the task? Best regards, Vitalii Tymchyshyn --089e0122acca9f37ba050e48c83b--