Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 25 May 2011 16:39:47 +0000 (UTC)
From: "Jonathan Ellis (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <1493684906.42644.1306341587363.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <301531132.28564.1305842987790.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Resolved] (CASSANDRA-2671) Not deleting compacted files
 even on out of disk space condition
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis resolved CASSANDRA-2671.
---------------------------------------

    Resolution: Not A Problem

{noformat}
INFO [CompactionExecutor:1] 2011-05-19 23:01:40,577 StorageService.java (line 1888) requesting GC to free disk space
INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,448 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1873
{noformat}

This is Cassandra cleaning up what it can when it is low on disk space. 

It looks like you had one sstable that was too big to "cleanup" even after deleting compacted files, but cleanup proceeded on smaller sstables which is why you had some left around to purge on restart.

> Not deleting compacted files even on out of disk space condition
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-2671
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2671
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.6
>         Environment: Linux 64-bit ubuntu. Installed via deb package (datastax distro)
>            Reporter: Marko Mikulicic
>
> I have a node consuming about 170 Gb and filling disk space:
>  INFO [CompactionExecutor:1] 2011-05-19 23:01:40,577 StorageService.java (line 1888) requesting GC to free disk space
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,448 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1873
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,449 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1872
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,467 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1584
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,481 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1609
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,492 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1789
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,505 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1663
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,539 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1769
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:01:51,560 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1794
> ERROR [CompactionExecutor:1] 2011-05-19 23:02:02,461 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main]
> java.io.IOException: disk full
>         at org.apache.cassandra.db.CompactionManager.doCleanupCompaction(CompactionManager.java:697)
>         at org.apache.cassandra.db.CompactionManager.access$500(CompactionManager.java:56)
>         at org.apache.cassandra.db.CompactionManager$2.call(CompactionManager.java:173)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:05:00,502 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1874
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:05:00,503 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1875
>  INFO [NonPeriodicTasks:1] 2011-05-19 23:05:00,504 SSTable.java (line 147) Deleted /var/lib/cassandra/data/dnet/body-f-1876
> Then it didn't continue to delete other files.
>  echo *Compacted | wc -l
> 309
> A lot of Compacted files remained (309). After restart they are deleted during startup.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira