Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 40125 invoked from network); 20 Jun 2010 14:32:14 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 20 Jun 2010 14:32:14 -0000 Received: (qmail 62394 invoked by uid 500); 20 Jun 2010 14:32:13 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 62364 invoked by uid 500); 20 Jun 2010 14:32:13 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 62356 invoked by uid 99); 20 Jun 2010 14:32:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 20 Jun 2010 14:32:13 +0000 X-ASF-Spam-Status: No, hits=2.8 required=10.0 tests=AWL,FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,TRACKER_ID,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jamesgolick@gmail.com designates 209.85.214.172 as permitted sender) Received: from [209.85.214.172] (HELO mail-iw0-f172.google.com) (209.85.214.172) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 20 Jun 2010 14:32:06 +0000 Received: by iwn2 with SMTP id 2so2648903iwn.31 for ; Sun, 20 Jun 2010 07:31:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=l+Kgt3qqxo53caao+hECqEh3JEW3eOf3SCcv6638uIg=; b=FCL8+9eRlEl5rhfX/q5DhFG+OrEU/hTKkwR7nFquvJyG5uzGoKUf8pBaHoBCOO4alP WUD1nLWZb35JzGVa5fUoDZzNzTmVELmQoszTiRINpsnYnl6SGcLuW+PJcc8UmAf/EOi7 NVjtzcY7hUktmXKPTf0YVwinfKEFY9ZtUOxLU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=HSoUD46HOxrdDt7IyZQ+dP1J2Xhz/0POrdcDbBM5TcQAgZijj9FvvdlH6tbQhd0xWX 1PR3wGyHi+FVHOh8klgaIn0zMFrPWXcpX8G9TyAm7p6ISwcxtC03g3WKlFwS+iNJjxGq NWPXSSbtKH0OnaQU0W+JG0D2aNAOQ/4k+8SNY= MIME-Version: 1.0 Received: by 10.231.124.40 with SMTP id s40mr3789540ibr.172.1277044305538; Sun, 20 Jun 2010 07:31:45 -0700 (PDT) Received: by 10.231.208.17 with HTTP; Sun, 20 Jun 2010 07:31:45 -0700 (PDT) In-Reply-To: References: Date: Sun, 20 Jun 2010 10:31:45 -0400 Message-ID: Subject: Re: Uneven distribution using RP From: James Golick To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016e646993a31ae010489770cba --0016e646993a31ae010489770cba Content-Type: text/plain; charset=ISO-8859-1 I ran cleanup on all of them and the distribution looked roughly even after that, but a couple of days later, it's looking pretty uneven. On Sun, Jun 20, 2010 at 10:21 AM, Jordan Pittier - Rezel wrote: > Hi, > Have you tried nodetool repair (or cleanup) on your nodes ? > > > On Sun, Jun 20, 2010 at 4:16 PM, James Golick wrote: > >> I just increased my cluster from 2 to 4 nodes, and RF=2 to RF=3, using RP. >> >> The tokens seem pretty even on the ring, but two of the nodes are far more >> heavily loaded than the others. I understand that there are a variety of >> possible reasons for this, but I'm wondering whether anybody has suggestions >> for now to tweak the tokens such that this problem is alleviated. Would it >> be better to just add 2 more nodes? >> >> Address Status Load Range >> Ring >> >> 170141183460469231731687303715884105728 >> 10.36.99.140 Up 61.73 GB >> 43733172796241720623128947447312912170 |<--| >> 10.36.99.134 Up 69.7 GB >> 85070591730234615865843651857942052864 | | >> 10.36.99.138 Up 54.08 GB >> 128813844387867495544257452469445200073 | | >> 10.36.99.136 Up 54.75 GB >> 170141183460469231731687303715884105728 |-->| >> >> > > --0016e646993a31ae010489770cba Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I ran cleanup on all of them and the distribution looked roughly even after= that, but a couple of days later, it's looking pretty uneven.

<= div class=3D"gmail_quote">On Sun, Jun 20, 2010 at 10:21 AM, Jordan Pittier = - Rezel <jordan@re= zel.net> wrote:
Hi,
Have you tried nodetool repair (or = cleanup) on your nodes ?


On Sun, Jun 20, 2010 at 4:16 PM, James Golick <jamesgolick@gmail.com> wrote:
I just increased my cluster from 2 to 4 node= s, and RF=3D2 to RF=3D3, using RP.

The tokens seem prett= y even on the ring, but two of the nodes are far more heavily loaded than t= he others. I understand that there are a variety of possible reasons for th= is, but I'm wondering whether anybody has suggestions for now to tweak = the tokens such that this problem is alleviated. Would it be better to just= add 2 more nodes?

Address =A0 =A0 =A0 Status =A0 =A0 Load =A0 =A0 =A0 =A0= =A0Range =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0Ring
=A0=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 170141183460469231731687303715884105728
10.36.99.140 =A0Up =A0 =A0 =A0 =A0 61.73 GB =A0 =A0 =A04373317279624= 1720623128947447312912170 =A0 =A0 |<--|
10.36.99.134 =A0Up =A0 =A0 =A0 =A0 69.7 GB =A0 =A0 =A0 850705917302346= 15865843651857942052864 =A0 =A0 | =A0 |
10.36.99.138 =A0Up =A0 = =A0 =A0 =A0 54.08 GB =A0 =A0 =A0128813844387867495544257452469445200073 =A0= =A0| =A0 |
10.36.99.136 =A0Up =A0 =A0 =A0 =A0 54.75 GB =A0 =A0 = =A0170141183460469231731687303715884105728 =A0 =A0|-->|
=A0


--0016e646993a31ae010489770cba--