cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (CASSANDRA-1858) File descriptors to sstables not closed (even for sstables that have been deleted due to compaction)
Date Tue, 14 Dec 2010 02:34:05 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis resolved CASSANDRA-1858.
---------------------------------------

    Resolution: Duplicate

if he saw it in rc1, it's probably CASSANDRA-1790 (fixed in rc2)

> File descriptors to sstables not closed (even for sstables that have been deleted due
to compaction)
> ----------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-1858
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1858
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0 rc 1
>         Environment: Ubuntu 8.04 with Java 1.6.0_22-b04
>            Reporter: Josep M. Blanquer
>
> The Cassandra process doesn't let go of filedescriptors to sstables.
> It was initially spotted due to the fact that the disk utilization was really high, while
the data dirs of sstables held much less (nicely compacted) data.
> Performing an "lsof" lists the Cassandra process having basically all open FD's pointing
to lots and lots of deleted files. In fact, I saw the "-1-Data.db" in the list...which tells
me that not even the first sstable was let go.
> Restarting the cassandra process obviously gets rid of the references. I've also manually
triggered a GC to try to make sure compaction is completed...to no avail. Holding FD's for
a high-throughput write servers is a big problem since it accumulates a huge amount of disk
space due to compactions..etc...so it's very easy to run out of space.
> There's nothing special in this setup, so I think it'd be easily reproducible. I get
this in a:
> * 4-node cluster, 1 Keyspace, 1 CF, RF=2
> * the cluster has basically been stood up, and blasted with inserts (now it has 200GB,
50 each)...but it doesn't get reads (I'm performing just loading data tests).
> * While the loading is happening, one can observe that the process keeps compacting and
creating new sstables...but never lets the FD's go.
> * even if I trigger compactions and cleanups...the disk util only continues to go up...i.e.,
all FD's are still open, plus the compated sstables are added.
> * at a particular node I see about 11 fd's for current sstables, and 1250 fd's pointing
to deleted sstables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message