Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
Sender: scode@scode.org
In-Reply-To: 
 <CAG00uyb70_H1APKiyZvDqf1us=TJxZfak41EkKxyT87cQ+2RRg@mail.gmail.com>
References: 
 <CAG00uybf1iroC8yUD2By0G2CUaxUj3effJ9u3dFxmGXvQZNO2w@mail.gmail.com>
	<CAO5xsd2fPoZiMmopUVXXD45HuggLssDOFrD16_syc1LjeK4_MA@mail.gmail.com>
	<CAG00uyb70_H1APKiyZvDqf1us=TJxZfak41EkKxyT87cQ+2RRg@mail.gmail.com>
Date: Fri, 19 Aug 2011 20:52:23 +0200
Message-ID: 
 <CAO5xsd0i5JP-tOSimQHBWyQxKK+8U632iJUvPWV5359v8mRyxg@mail.gmail.com>
Subject: Re: nodetool repair caused high disk space usage
From: Peter Schuller <peter.schuller@infidyne.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

> There were few Compacted files.=C2=A0 I thought that might have been the =
cause,
> but it wasn't it.=C2=A0 We have a CF that is 23GB, and while repair is ru=
nning,
> there are multiple instances of that CF created along with other CFs.

To confirm - are you saying the data directory size is huge, but the
live size as reported by nodetool ring and nodetool info does NOT
reflect this inflated size?

What files *do* you have in the data directory? Any left-over *tmp*
files for example?

Are you sure you're only running a single repair at a time? (Sorry if
this was covered, I did a quick swipe through thread history because I
was unsure whether I was confusing two different threads, and I don't
think so.)

The question is what's taking the space. If it's sstables, they really
should be either compacted onces that are marked for deletion but
being retained, or "live" sstables in which case they should show up
as load in nodetool.

What else... maybe streams are being re-tried from the source nodes
and the disk space is coming from a bunch of half-finished streams of
the same data. But if so, those should be *tmp* files IIRC.

I'm just wildly speculation, but it would be nice to get to the bottom of t=
his.

--=20
/ Peter Schuller (@scode on twitter)