Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 33279 invoked from network); 4 Jun 2010 16:50:43 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 4 Jun 2010 16:50:43 -0000 Received: (qmail 53824 invoked by uid 500); 4 Jun 2010 16:50:41 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 53810 invoked by uid 500); 4 Jun 2010 16:50:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 53802 invoked by uid 99); 4 Jun 2010 16:50:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jun 2010 16:50:41 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of isoboroff@gmail.com designates 209.85.160.172 as permitted sender) Received: from [209.85.160.172] (HELO mail-gy0-f172.google.com) (209.85.160.172) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jun 2010 16:50:34 +0000 Received: by gyh4 with SMTP id 4so1095463gyh.31 for ; Fri, 04 Jun 2010 09:50:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=22RSLn+ED88nVJzWlYC4YS8y7m24N/h0F9l653cuIq4=; b=GOMXUSLgIi66LLhw82CL71lxdX0Y+oV0U8koIs81bcPszFCR3GF39TwzD+omRWSMn4 wUHPCrZnaqKS7EzP8ypjrM5t1RpIWqU/VAE+PXmzg2UCeUwdD71AbISJJHFLGxRiG+cB G/DsTSh1JEAHXX++i//L38w9375AjYIw7gkQc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=VS34ixt3hHStCRvZekjB37JJ1citDRTrdL5InekRx4PfnlMsFJ7bfkUVusIY7Hfncv 9IM1kaDlM85R31KhNY3wCW07abfPanzN1DTS+vAtwItVt6aW1eyKW6qsvKBbstiRW7t7 GecK0VmqPNQZF/mur9bXtaeQsJWLfmKzr09Xw= MIME-Version: 1.0 Received: by 10.229.214.195 with SMTP id hb3mr357807qcb.292.1275670213637; Fri, 04 Jun 2010 09:50:13 -0700 (PDT) Received: by 10.229.44.21 with HTTP; Fri, 4 Jun 2010 09:50:13 -0700 (PDT) In-Reply-To: References: Date: Fri, 4 Jun 2010 12:50:13 -0400 Message-ID: Subject: Re: Handling disk-full scenarios From: Ian Soboroff To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016362842e8ef39530488371d08 X-Virus-Checked: Checked by ClamAV on apache.org --0016362842e8ef39530488371d08 Content-Type: text/plain; charset=ISO-8859-1 Story continued, in hopes this experience is useful to someone... I shut down the node, removed the huge file, restarted the node, and told everybody to repair. Two days later, AE stages are still running. Ian On Thu, Jun 3, 2010 at 2:21 AM, Jonathan Ellis wrote: > this is why JBOD configuration is contraindicated for cassandra. > http://wiki.apache.org/cassandra/CassandraHardware > > On Tue, Jun 1, 2010 at 1:08 PM, Ian Soboroff wrote: > > My nodes have 5 disks and are using them separately as data disks. The > > usage on the disks is not uniform, and one is nearly full. Is there some > > way to manually balance the files across the disks? Pretty much anything > > done via nodetool incurs an anticompaction with obviously fails. system/ > is > > not the problem, it's in my data's keyspace. > > > > Ian > > > > > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of Riptano, the source for professional Cassandra support > http://riptano.com > --0016362842e8ef39530488371d08 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Story continued, in hopes this experience is useful to someone...

I = shut down the node, removed the huge file, restarted the node, and told eve= rybody to repair.=A0 Two days later, AE stages are still running.

Ia= n

On Thu, Jun 3, 2010 at 2:21 AM, Jonathan Ell= is <jbellis@gmail= .com> wrote:
this is why JBOD configuration is contraindicated for cassandra.
http://wiki.apache.org/cassandra/CassandraHardware

On Tue, Jun 1, 2010 at 1:08 PM, Ian Soboroff <isoboroff@gmail.com> wrote:
> My nodes have 5 disks and are = using them separately as data disks.=A0 The
> usage on the disks is not uniform, and one is nearly full.=A0 Is there= some
> way to manually balance the files across the disks?=A0 Pretty much any= thing
> done via nodetool incurs an anticompaction with obviously fails.=A0 sy= stem/ is
> not the problem, it's in my data's keyspace.
>
> Ian
>
>



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

--0016362842e8ef39530488371d08--