incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: sstable size change
Date Mon, 29 Jul 2013 08:02:24 GMT
> I still would have expected my original alter command to at least have thrown an error.
 I assume I should open a bug for this?
Sounds like a good idea. 

Please describe the version and query you first used. 

Thanks for taking the time to update the thread. 

Cheers


-----------------
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 27/07/2013, at 12:57 AM, Keith Wright <kwright@nanigans.com> wrote:

> FYI.  It appears that the proper command in CQL3 is the following:
> 
> alter table cookie_user_lookup with compaction={'sstable_size_in_mb': '256', 'class':
'LeveledCompactionStrategy'};
> 
> I still would have expected my original alter command to at least have thrown an error.
 I assume I should open a bug for this?
> 
> alter table shard_user_lookup with compaction_strategy_options = {'sstable_size_in_mb':256};
> 
> 
> From: Keith Wright <kwright@nanigans.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Thursday, July 25, 2013 9:14 AM
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>, Wei Zhu <wz1975@yahoo.com>
> Subject: Re: sstable size change
> 
> Unfortunately the table in question is a CQL3 table so cassandra-cli will not output
its describe:
> 
> WARNING: CQL3 tables are intentionally omitted from 'describe' output.
> See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.
> 
> However, I did figure out that apparently I was not setting the change properly.  I was
using the following alter via CQL3:
> 
> alter table shard_user_lookup with compaction_strategy_options = {'sstable_size_in_mb':256};
> 
> But when I did a describe, I did not see the change.   
> 
> CREATE TABLE shard_user_lookup (
>   shard_user_id int,
>   shard text,
>   creation_time timestamp,
>   status int,
>   user_id timeuuid,
>   PRIMARY KEY (shard_user_id, shard)
> ) WITH
>   bloom_filter_fp_chance=0.100000 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.000000 AND
>   gc_grace_seconds=86400 AND
>   read_repair_chance=0.100000 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'class': 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 'sstable_compression':
'LZ4Compressor'};
> 
> If I then execute the following via CQL2, I do see the property.  Seems odd to me?
> 
> alter table shard_user_lookup with compaction_strategy_options:sstable_size_in_mb=256;
> 
> CREATE TABLE shard_user_lookup (
>   shard_user_id int,
>   shard text,
>   creation_time timestamp,
>   status int,
>   user_id timeuuid,
>   PRIMARY KEY (shard_user_id, shard)
> ) WITH
>   bloom_filter_fp_chance=0.100000 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.000000 AND
>   gc_grace_seconds=86400 AND
>   read_repair_chance=0.100000 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '256', 'class': 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 'sstable_compression':
'LZ4Compressor'};
> 
> 
> From: Wei Zhu <wz1975@yahoo.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>, Wei Zhu <wz1975@yahoo.com>
> Date: Wednesday, July 24, 2013 8:49 PM
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: sstable size change
> 
> what is output of show keyspaces from cassandra-cli, did you see the new value?
> 
>   Compaction Strategy: org.apache.cassandra.db.compaction.LeveledCompactionStrategy
>       Compaction Strategy Options:
>         sstable_size_in_mb: XXX
> 
> From: Keith Wright <kwright@nanigans.com>
> To: "user@cassandra.apache.org" <user@cassandra.apache.org> 
> Sent: Wednesday, July 24, 2013 3:44 PM
> Subject: Re: sstable size change
> 
> Hi all,
> 
>    This morning I increased the SSTable size for one of my LCS via an alter command and
saw at least one compaction run (I did not trigger a compaction via nodetool nor upgrades
stables nor removing the .json file).  But so far my data sizes appear the same at the default
5 MB (see below for output of ls –Sal as well as relevant portion of cfstats).   Is this
expected?  I was hoping to see at least one file at the new 256 MB size I set.
> 
> Thanks
> 
> SSTable count: 4965
> SSTables in each level: [0, 10, 112/100, 1027/1000, 3816, 0, 0, 0]
> Space used (live): 29062393142
> Space used (total): 29140547702
> Number of Keys (estimate): 195103104
> Memtable Columns Count: 441483
> Memtable Data Size: 205486218
> Memtable Switch Count: 243
> Read Count: 154226729
> 
> -rw-rw-r--  1 cassandra cassandra 5247564 Jul 18 01:33 users-shard_user_lookup-ib-97153-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247454 Jul 23 02:59 users-shard_user_lookup-ib-109063-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247421 Jul 20 14:58 users-shard_user_lookup-ib-103127-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247415 Jul 17 13:56 users-shard_user_lookup-ib-95761-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247379 Jul 21 02:44 users-shard_user_lookup-ib-104718-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247346 Jul 21 21:54 users-shard_user_lookup-ib-106280-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247242 Jul  3 19:41 users-shard_user_lookup-ib-66049-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247235 Jul 21 02:44 users-shard_user_lookup-ib-104737-Data.db
> -rw-rw-r--  1 cassandra cassandra 5247233 Jul 20 14:58 users-shard_user_lookup-ib-103169-Data.db
> 
> 
> From: sankalp kohli <kohlisankalp@gmail.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Tuesday, July 23, 2013 3:04 PM
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: sstable size change
> 
> "Will Cassandra force any newly compacted files to my new setting as compactions are
naturally triggered"
> Yes. Let it compact and increase in size. 
> 
> 
> On Tue, Jul 23, 2013 at 9:38 AM, Robert Coli <rcoli@eventbrite.com> wrote:
>> On Tue, Jul 23, 2013 at 6:48 AM, Keith Wright <kwright@nanigans.com> wrote:
>>> Can you elaborate on what you mean by "let it take its own course organically"?
 Will Cassandra force any newly compacted files to my new setting as compactions are naturally
triggered?
>> 
>> You see, when two (or more!) SSTables love each other very much, they sometimes decide
they want to compact together..
>> 
>> But seriously, "yes." If you force all existing SSTables to level 0, it is as if
you just flushed them all. Level compaction then does a whole lot of compaction, using the
active table size.
>> 
>> =Rob
> 
> 
> 


Mime
View raw message