incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wright <kwri...@nanigans.com>
Subject Re: sstable size change
Date Thu, 25 Jul 2013 13:14:45 GMT
Unfortunately the table in question is a CQL3 table so cassandra-cli will not output its describe:

WARNING: CQL3 tables are intentionally omitted from 'describe' output.
See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.

However, I did figure out that apparently I was not setting the change properly.  I was using
the following alter via CQL3:

alter table shard_user_lookup with compaction_strategy_options = {'sstable_size_in_mb':256};

But when I did a describe, I did not see the change.

CREATE TABLE shard_user_lookup (
  shard_user_id int,
  shard text,
  creation_time timestamp,
  status int,
  user_id timeuuid,
  PRIMARY KEY (shard_user_id, shard)
) WITH
  bloom_filter_fp_chance=0.100000 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.000000 AND
  gc_grace_seconds=86400 AND
  read_repair_chance=0.100000 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  compaction={'class': 'LeveledCompactionStrategy'} AND
  compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 'sstable_compression': 'LZ4Compressor'};

If I then execute the following via CQL2, I do see the property.  Seems odd to me?

alter table shard_user_lookup with compaction_strategy_options:sstable_size_in_mb=256;

CREATE TABLE shard_user_lookup (
  shard_user_id int,
  shard text,
  creation_time timestamp,
  status int,
  user_id timeuuid,
  PRIMARY KEY (shard_user_id, shard)
) WITH
  bloom_filter_fp_chance=0.100000 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.000000 AND
  gc_grace_seconds=86400 AND
  read_repair_chance=0.100000 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  compaction={'sstable_size_in_mb': '256', 'class': 'LeveledCompactionStrategy'} AND
  compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 'sstable_compression': 'LZ4Compressor'};


From: Wei Zhu <wz1975@yahoo.com<mailto:wz1975@yahoo.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>,
Wei Zhu <wz1975@yahoo.com<mailto:wz1975@yahoo.com>>
Date: Wednesday, July 24, 2013 8:49 PM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: sstable size change

what is output of show keyspaces from cassandra-cli, did you see the new value?

  Compaction Strategy: org.apache.cassandra.db.compaction.LeveledCompactionStrategy
      Compaction Strategy Options:
        sstable_size_in_mb: XXX

________________________________
From: Keith Wright <kwright@nanigans.com<mailto:kwright@nanigans.com>>
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Sent: Wednesday, July 24, 2013 3:44 PM
Subject: Re: sstable size change

Hi all,

   This morning I increased the SSTable size for one of my LCS via an alter command and saw
at least one compaction run (I did not trigger a compaction via nodetool nor upgrades stables
nor removing the .json file).  But so far my data sizes appear the same at the default 5 MB
(see below for output of ls –Sal as well as relevant portion of cfstats).   Is this expected?
 I was hoping to see at least one file at the new 256 MB size I set.

Thanks

SSTable count: 4965
SSTables in each level: [0, 10, 112/100, 1027/1000, 3816, 0, 0, 0]
Space used (live): 29062393142
Space used (total): 29140547702
Number of Keys (estimate): 195103104
Memtable Columns Count: 441483
Memtable Data Size: 205486218
Memtable Switch Count: 243
Read Count: 154226729

-rw-rw-r--  1 cassandra cassandra 5247564 Jul 18 01:33 users-shard_user_lookup-ib-97153-Data.db
-rw-rw-r--  1 cassandra cassandra 5247454 Jul 23 02:59 users-shard_user_lookup-ib-109063-Data.db
-rw-rw-r--  1 cassandra cassandra 5247421 Jul 20 14:58 users-shard_user_lookup-ib-103127-Data.db
-rw-rw-r--  1 cassandra cassandra 5247415 Jul 17 13:56 users-shard_user_lookup-ib-95761-Data.db
-rw-rw-r--  1 cassandra cassandra 5247379 Jul 21 02:44 users-shard_user_lookup-ib-104718-Data.db
-rw-rw-r--  1 cassandra cassandra 5247346 Jul 21 21:54 users-shard_user_lookup-ib-106280-Data.db
-rw-rw-r--  1 cassandra cassandra 5247242 Jul  3 19:41 users-shard_user_lookup-ib-66049-Data.db
-rw-rw-r--  1 cassandra cassandra 5247235 Jul 21 02:44 users-shard_user_lookup-ib-104737-Data.db
-rw-rw-r--  1 cassandra cassandra 5247233 Jul 20 14:58 users-shard_user_lookup-ib-103169-Data.db


From: sankalp kohli <kohlisankalp@gmail.com<mailto:kohlisankalp@gmail.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Tuesday, July 23, 2013 3:04 PM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: sstable size change

"Will Cassandra force any newly compacted files to my new setting as compactions are naturally
triggered"
Yes. Let it compact and increase in size.


On Tue, Jul 23, 2013 at 9:38 AM, Robert Coli <rcoli@eventbrite.com<mailto:rcoli@eventbrite.com>>
wrote:
On Tue, Jul 23, 2013 at 6:48 AM, Keith Wright <kwright@nanigans.com<mailto:kwright@nanigans.com>>
wrote:
Can you elaborate on what you mean by "let it take its own course organically"?  Will Cassandra
force any newly compacted files to my new setting as compactions are naturally triggered?

You see, when two (or more!) SSTables love each other very much, they sometimes decide they
want to compact together..

But seriously, "yes." If you force all existing SSTables to level 0, it is as if you just
flushed them all. Level compaction then does a whole lot of compaction, using the active table
size.

=Rob




Mime
View raw message