Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of cryptofive@gmail.com
 designates 209.85.214.172 as permitted sender)
MIME-Version: 1.0
Date: Mon, 23 Apr 2012 18:50:07 -0700
Message-ID: 
 <CAEAJz-pw6kkvjX6chO25_jkUFQ12BhkvCjzkzLkUV2j_LGinOQ@mail.gmail.com>
Subject: Cassandra dying when gets many deletes
From: crypto five <cryptofive@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=e89a8fb1f81a6e87ca04be62f98e

--e89a8fb1f81a6e87ca04be62f98e
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I have 50 millions of rows in column family on 4G RAM box. I allocatedf 2GB
to cassandra.
I have program which is traversing this CF and cleaning some data there, it
generates about 20k delete statements per second.
After about of 3 millions deletions cassandra stops responding to queries:
it doesn't react to CLI, nodetool etc.
I see in the logs that it tries to free some memory but can't even if I
wait whole day.
Also I see following in  the logs:

INFO [ScheduledTasks:1] 2012-04-23 18:38:13,333 StorageService.java (line
2647) Unable to reduce heap usage since there are no dirty column families

When I am looking at memory dump I see that memory goes to
ConcurrentSkipListMap(10%), HeapByteBuffer(13%), DecoratedKey(6%),
int[](6%), BigInteger(8.2%), ConcurrentSkipListMap$HeadIndex(7.2%),
ColumnFamily(6.5%), ThreadSafeSortedColumns(13.7%), long[](5.9%).

What can I do to make cassandra stop dying?
Why it can't free the memory?
Any ideas?

Thank you.

--e89a8fb1f81a6e87ca04be62f98e
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,<div><br></div><div>I have 50 millions of rows in column family on 4G RA=
M box. I allocatedf 2GB to cassandra.</div><div>I have program which is tra=
versing this CF and cleaning some data there, it generates about 20k delete=
 statements per second.</div>
<div>After about of 3 millions deletions cassandra stops responding to quer=
ies: it doesn&#39;t react to CLI, nodetool etc.</div><div>I see in the logs=
 that it tries to free some memory but can&#39;t even if I wait whole day.=
=A0</div>
<div>Also I see following in =A0the logs:</div><div><br></div><div>INFO [Sc=
heduledTasks:1] 2012-04-23 18:38:13,333 StorageService.java (line 2647) Una=
ble to reduce heap usage since there are no dirty column families</div><div=
>
<br></div><div>When I am looking at memory dump I see that memory goes to C=
oncurrentSkipListMap(10%), HeapByteBuffer(13%), DecoratedKey(6%), int[](6%)=
, BigInteger(8.2%), ConcurrentSkipListMap$HeadIndex(7.2%), ColumnFamily(6.5=
%), ThreadSafeSortedColumns(13.7%), long[](5.9%).</div>
<div><br></div><div>What can I do to make cassandra stop dying?=A0</div><di=
v>Why it can&#39;t free the memory?</div><div>Any ideas?</div><div><br></di=
v><div>Thank you.</div>

--e89a8fb1f81a6e87ca04be62f98e--