Return-Path: X-Original-To: apmail-helix-user-archive@minotaur.apache.org Delivered-To: apmail-helix-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D9DFF1825F for ; Thu, 25 Jun 2015 06:13:48 +0000 (UTC) Received: (qmail 44027 invoked by uid 500); 25 Jun 2015 06:13:48 -0000 Delivered-To: apmail-helix-user-archive@helix.apache.org Received: (qmail 43981 invoked by uid 500); 25 Jun 2015 06:13:48 -0000 Mailing-List: contact user-help@helix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@helix.apache.org Delivered-To: mailing list user@helix.apache.org Received: (qmail 43971 invoked by uid 99); 25 Jun 2015 06:13:48 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jun 2015 06:13:48 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 1CE05D014B for ; Thu, 25 Jun 2015 06:13:48 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.88 X-Spam-Level: ** X-Spam-Status: No, score=2.88 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id bOL3XuTu6Pur for ; Thu, 25 Jun 2015 06:13:47 +0000 (UTC) Received: from mail-qk0-f182.google.com (mail-qk0-f182.google.com [209.85.220.182]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 6959E27D76 for ; Thu, 25 Jun 2015 06:13:46 +0000 (UTC) Received: by qkhu186 with SMTP id u186so33745308qkh.0 for ; Wed, 24 Jun 2015 23:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=B+DboSY14QlVHVwVBAEx0D9GOVGLpn8WyZq0nJRgwbo=; b=ncLxzixXypTHhxTd0qmbgxl+hKKkg6/WtiT2no6gAbJPsBEngnreAiIeL/NqeejKGj XCeaRIQ7TmTT240z4ruBcMV+8rrT3Enuv69TFklB0YVkjdXP8EQhaGJF4NimgUa5NPR4 4gPORlCcHjRGMuu0jjoSXfxhodWXiFWJ/Jcu+qpkq4mL3gf+o/BrexkZWIMznGHPRLEj I44i2GJ9DTpvMJmid8gYGQ6Ni3siPhA5r01BXm9O2qLpjRJ/P4DK5Zba4mybt73RZ8ob G1gEyNLS+fJIku/qSXwhBCpXth/dw2o0COXt4xGwC0sLm0rlPklT8gucAgahw4SsI3/1 B50Q== MIME-Version: 1.0 X-Received: by 10.140.46.75 with SMTP id j69mr56452216qga.17.1435212819021; Wed, 24 Jun 2015 23:13:39 -0700 (PDT) Received: by 10.96.101.36 with HTTP; Wed, 24 Jun 2015 23:13:38 -0700 (PDT) In-Reply-To: References: Date: Wed, 24 Jun 2015 23:13:38 -0700 Message-ID: Subject: Re: Customized rebalancer From: Changgeng Li To: kishore g Cc: user@helix.apache.org Content-Type: multipart/alternative; boundary=001a113a68044147f805195186db --001a113a68044147f805195186db Content-Type: text/plain; charset=UTF-8 yes, it's always null. actually I checked the code that parameter was never set. https://github.com/apache/helix/blob/acd902e2433f65e9864ccf49fcd1a04c36b1f206/helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java#L222 On Wed, Jun 24, 2015 at 10:41 PM, kishore g wrote: > I see that you haven't subscribed to mailing list yet. Please do that my > sending an email to user-subscribe@helix.apache.org. > > I am also available on #apachehelix irc if you have additional questions. > http://helix.apache.org/IRC.html > > > ---------- Forwarded message ---------- > From: kishore g > Date: Wed, Jun 24, 2015 at 10:37 PM > Subject: Re: Customized rebalancer > To: "user@helix.apache.org" > > > Hi Changgeng, > > I think the first invocation will always be null, subsequent invocations > should provide you the previous resource assignment. Are you saying its > always null?. > > thanks, > Kishore G > > On Wed, Jun 24, 2015 at 3:42 PM, Changgeng Li > wrote: > >> Hello, >> >> We have a use case that when adding or removing a new node, we hope to >> minimize the shuffle of partitions between nodes. I'm trying to implement a >> customized rebalancer calculating the new resource assignment based on the >> previous resource assignment. When a new node is added, just move some >> partitions from existing node to the new node, and when a node is down, >> move the partitions on this node to the other nodes. Partitions would not >> move between two nodes if the both status are not changed. >> >> The documentation says: >> In rebalance(), ... the third is the output of the previous invocation >> of this method (if supported) ... >> >> During my testing I found the previous resource assignment passed in is >> always null. my question is what "if supported" means here? Does the >> rebalancer need to support something? >> >> >> Any other suggestions are also welcome. >> >> Thanks, >> Changgeng >> > > > --001a113a68044147f805195186db Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
=
On Wed, Jun 24, 2015 at 10:41 PM, kishore g = <g.kishore@gmail.com> wrote:
I see that you haven't subscribed to mailing list= yet. Please do that my sending an email to user-subscribe@helix.apache.org.<= div>
I am also available on #apachehelix irc if you have addi= tional questions. http://helix.apache.org/IRC.html


---------- Forwarded message ----------
From: kishore g <g.kishore@gmail.com>
Date: Wed, Jun 24, 2015 at 10:37 PM
Subject: Re: Customized reb= alancer
To: "user@helix.apache.org" <user@helix.apache.org>


Hi Changgeng,

I think the first invocation will= always be null, subsequent invocations should provide you the previous res= ource assignment. Are you saying its always null?.

thanks,
Kishore G

On Wed, Jun 24, 2015 at 3:42 PM, Changgeng= Li <li.changgeng@gmail.com> wrote:
Hello,

We have a use case that wh= en adding or removing a new node, we hope to minimize the shuffle of partit= ions between nodes. I'm trying to implement a customized rebalancer cal= culating the new resource assignment based on the previous resource assignm= ent. When a new node is added, just move some partitions from existing node= to the new node, and when a node is down, move the partitions on this node= to the other nodes. Partitions would not move between two nodes if the bot= h status are not changed.

The documentation says:
In = rebalance(), ... the third is the output of the=20 previous invocation of this method (if supported) ...

Dur= ing my testing I found the previous resource assignment passed in is always= null. my question is what "if supported" means here? Does the re= balancer need to support something?


Any other suggest= ions are also welcome.

Thanks,
Changgeng



--001a113a68044147f805195186db--