cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom van der Woerdt (JIRA)" <>
Subject [jira] [Created] (CASSANDRA-12114) Cassandra startup takes an hour because of
Date Thu, 30 Jun 2016 14:28:10 GMT
Tom van der Woerdt created CASSANDRA-12114:

             Summary: Cassandra startup takes an hour because of
                 Key: CASSANDRA-12114
             Project: Cassandra
          Issue Type: Bug
          Components: Core
            Reporter: Tom van der Woerdt

A Cassandra cluster of ours has nodes with up to 4TB of data, in a single table using leveled
compaction having 200k files. While upgrading from 2.2.6 to 3.0.7 we noticed that it took
a while to restart a node. And with "a while" I mean we measured it at more than 60 minutes.

jstack shows something interesting :
"main" #1 prio=5 os_prio=0 tid=0x00007f30db0ea400 nid=0xdb22 runnable [0x00007f30de122000]
   java.lang.Thread.State: RUNNABLE
    at Method)
    at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(
    at org.apache.cassandra.service.CassandraDaemon.setup(
    at org.apache.cassandra.service.CassandraDaemon.activate(
    at org.apache.cassandra.service.CassandraDaemon.main(

Going by the source of File.listFiles, it puts every file in a directory into an array and
*then* applies the filter.

This is actually a known Java issue from 1999:;jsessionid=db7fcf25bcce13541c4289edeb4?bug_id=4285834
-- their "solution" was to introduce new APIs in JRE7. I guess that makes listFiles deprecated
for larger directories (like when using LeveledCompactionStrategy).

tl;dr: because Cassandra uses, service startup can take an hour for
larger data sets.

This message was sent by Atlassian JIRA

View raw message