cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rana Aich <aichr...@gmail.com>
Subject Strange Behavior : Commitlog data is not flushed
Date Wed, 06 Oct 2010 18:51:01 GMT
Hello Experts,

I see a queer behavior from on of the Cassandra nodes in my cluster where
the data is not flushed off Commitlogs and the Commitlog file grows in
number. I was inserting the data into the cluster and since yesterday this
node had more than 900 commitlog files.

-rw-r--r-- 1 dev dev  134218072 2010-10-05 15:11 CommitLog-1286218976629.log
-rw-r--r-- 1 dev dev  134217909 2010-10-05 15:11 CommitLog-1286316625590.log
-rw-r--r-- 1 dev dev  134217872 2010-10-05 15:15 CommitLog-1286316673050.log
-rw-r--r-- 1 dev dev  134217957 2010-10-05 15:19 CommitLog-1286316917618.log
-rw-r--r-- 1 dev dev  134217764 2010-10-05 15:21 CommitLog-1286317178922.log
-rw-r--r-- 1 dev dev  134217871 2010-10-05 15:24 CommitLog-1286317311372.log
-rw-r--r-- 1 dev dev  134218029 2010-10-05 15:25 CommitLog-1286317446787.log
-rw-r--r-- 1 dev dev  134218065 2010-10-05 15:26 CommitLog-1286317506892.log
-rw-r--r-- 1 dev dev  134218100 2010-10-05 15:27 CommitLog-1286317567927.log
-rw-r--r-- 1 dev dev  134218020 2010-10-05 15:29 CommitLog-1286317634668.log
-rw-r--r-- 1 dev dev  134217922 2010-10-05 15:31 CommitLog-1286317741576.log
-rw-r--r-- 1 dev dev  134217944 2010-10-05 15:32 CommitLog-1286317887655.log
-rw-r--r-- 1 dev dev  134217961 2010-10-05 15:33 CommitLog-1286317968395.log
-rw-r--r-- 1 dev dev  134217926 2010-10-05 15:34 CommitLog-1286318033241.log
-rw-r--r-- 1 dev dev  134218045 2010-10-05 15:36 CommitLog-1286318097727.log
-rw-r--r-- 1 dev dev  134218027 2010-10-05 15:37 CommitLog-1286318190654.log
-rw-r--r-- 1 dev dev  134218060 2010-10-05 15:38 CommitLog-1286318277987.log
-rw-r--r-- 1 dev dev  134218055 2010-10-05 15:41 CommitLog-1286318338561.log
-rw-r--r-- 1 dev dev  134218069 2010-10-05 15:41 CommitLog-1286318461180.log
-rw-r--r-- 1 dev dev  134217791 2010-10-05 15:43 CommitLog-1286318519193.log
-rw-r--r-- 1 dev dev  134218066 2010-10-05 15:44 CommitLog-1286318609960.log
-rw-r--r-- 1 dev dev  134218093 2010-10-05 15:45 CommitLog-1286318692656.log

It seems that commitlog files spawned every minute and when it grew over 900
files the Cassandra node went down.

Now I'm trying to start my node but it has started to read all of those
Commitlog files and takes huge amount of time to restart.

Can anyone throw some light on this unusual behavior. Till yesterday before
starting the data insertion the node had almost 550GB of data. Incidentally
I did stop another node prior to inserting the data because it was 100%
full. I'm using RandomPartitioner with Initial Tokens.

Thanking all in advance.

Rana

Mime
View raw message