incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weijun Li <weiju...@gmail.com>
Subject nodetool-compact duplicated data files again and again
Date Wed, 17 Mar 2010 21:28:23 GMT
I'm testing the ExpiringColumn patch in 0.6-beta2, inserted 26GB data with
TTL, after columns have expired I use get_slice to verify that no columns
can be retrieved. When I run "nodetool compact" I think all data should be
gone. But the problem is:

1) After the first nodetool-comact, Cassandra duplicate data files to
data-377* and then nothing happened. Total files size become 52GB. Some 0
bytes *.Compacted files got generated.
2) After the second nodetool-compact, Cassandra again generated data-378*.
Now I got 77GB data file that contains no valid columns. (See the list at
the end)
3) Now I decided to run nodetool-clean and it ended up with 50GB data files
like:
total 53717104
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 17:25 data-378-Compacted
-rw-rw-r-- 1 cassandra cassandra 25563592504 Mar 17 16:25 data-378-Data.db
-rw-rw-r-- 1 cassandra cassandra    54326245 Mar 17 16:25 data-378-Filter.db
-rw-rw-r-- 1 cassandra cassandra  1871937928 Mar 17 16:25 data-378-Index.db
-rw-rw-r-- 1 cassandra cassandra 25563592504 Mar 17 17:25 data-379-Data.db
-rw-rw-r-- 1 cassandra cassandra    27163165 Mar 17 17:25 data-379-Filter.db
-rw-rw-r-- 1 cassandra cassandra  1871937928 Mar 17 17:25 data-379-Index.db

Any idea about what's going on here? I guess cleanup will remove all columns
and don't belong this node but compact will remove all deleted columns then
merge small files into a big one. What exactly are the differences between
cleanup and compact?

-Weijun

total 80615576
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 15:27 data-327-Compacted
-rw-rw-r-- 1 cassandra cassandra 21013367426 Mar 16 17:43 data-327-Data.db
-rw-rw-r-- 1 cassandra cassandra    44660005 Mar 16 17:43 data-327-Filter.db
-rw-rw-r-- 1 cassandra cassandra  1538760208 Mar 16 17:43 data-327-Index.db
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 15:27 data-363-Compacted
-rw-rw-r-- 1 cassandra cassandra  2767150915 Mar 16 17:46 data-363-Data.db
-rw-rw-r-- 1 cassandra cassandra     5890885 Mar 16 17:46 data-363-Filter.db
-rw-rw-r-- 1 cassandra cassandra   202590655 Mar 16 17:46 data-363-Index.db
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 15:27 data-370-Compacted
-rw-rw-r-- 1 cassandra cassandra  1383745492 Mar 16 17:47 data-370-Data.db
-rw-rw-r-- 1 cassandra cassandra     2947045 Mar 16 17:47 data-370-Filter.db
-rw-rw-r-- 1 cassandra cassandra   101350867 Mar 16 17:47 data-370-Index.db
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 15:27 data-375-Compacted
-rw-rw-r-- 1 cassandra cassandra   345870869 Mar 16 17:50 data-375-Data.db
-rw-rw-r-- 1 cassandra cassandra      736405 Mar 16 17:50 data-375-Filter.db
-rw-rw-r-- 1 cassandra cassandra    25315970 Mar 16 17:50 data-375-Index.db
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 15:27 data-376-Compacted
-rw-rw-r-- 1 cassandra cassandra    53457802 Mar 16 18:52 data-376-Data.db
-rw-rw-r-- 1 cassandra cassandra      113853 Mar 16 18:52 data-376-Filter.db
-rw-rw-r-- 1 cassandra cassandra     3920228 Mar 16 18:52 data-376-Index.db
-rw-rw-r-- 1 cassandra cassandra           0 Mar 17 16:25 data-377-Compacted
-rw-rw-r-- 1 cassandra cassandra 25563592504 Mar 17 15:27 data-377-Data.db
-rw-rw-r-- 1 cassandra cassandra    54327685 Mar 17 15:27 data-377-Filter.db
-rw-rw-r-- 1 cassandra cassandra  1871937928 Mar 17 15:27 data-377-Index.db
-rw-rw-r-- 1 cassandra cassandra 25563592504 Mar 17 16:25 data-378-Data.db
-rw-rw-r-- 1 cassandra cassandra    54326245 Mar 17 16:25 data-378-Filter.db
-rw-rw-r-- 1 cassandra cassandra  1871937928 Mar 17 16:25 data-378-Index.db

Mime
View raw message