Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AB84EC491 for ; Mon, 29 Jul 2013 08:03:02 +0000 (UTC) Received: (qmail 80482 invoked by uid 500); 29 Jul 2013 08:02:58 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 80450 invoked by uid 500); 29 Jul 2013 08:02:58 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 80441 invoked by uid 99); 29 Jul 2013 08:02:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jul 2013 08:02:56 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a46.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jul 2013 08:02:50 +0000 Received: from homiemail-a46.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a46.g.dreamhost.com (Postfix) with ESMTP id 9B03D3E405C for ; Mon, 29 Jul 2013 01:02:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :content-type:message-id:mime-version:subject:date:references:to :in-reply-to; s=thelastpickle.com; bh=4aMxFcOz1vIRglSXVxtV4re2jA 8=; b=SO7HK+7wXqXzm/B5e4EMdYyxzfsJdKOKQx0j6F/Aly2ifkU4ERCt06pP+r ZIpLP3QRkd4nb/76kufODCi8RFmdCrNLiCT5oTagQAfI0tsB0JVj6P56BLcwo3Zz X36ShLlH6zWGYnMBCSiczDoqeAG0ecaZE33xXyJUcnzOuuwd4= Received: from [172.16.1.7] (unknown [203.86.207.101]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a46.g.dreamhost.com (Postfix) with ESMTPSA id 8AD4A3E405B for ; Mon, 29 Jul 2013 01:02:27 -0700 (PDT) From: aaron morton Content-Type: multipart/alternative; boundary="Apple-Mail=_9DA298A6-CEC3-4863-9769-AC168216A3A7" Message-Id: <74F95338-3423-4000-A971-4304D1F4716F@thelastpickle.com> Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: sstable size change Date: Mon, 29 Jul 2013 20:02:24 +1200 References: To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1508) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_9DA298A6-CEC3-4863-9769-AC168216A3A7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > I still would have expected my original alter command to at least have = thrown an error. I assume I should open a bug for this? Sounds like a good idea.=20 Please describe the version and query you first used.=20 Thanks for taking the time to update the thread.=20 Cheers ----------------- Aaron Morton Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 27/07/2013, at 12:57 AM, Keith Wright wrote: > FYI. It appears that the proper command in CQL3 is the following: >=20 > alter table cookie_user_lookup with compaction=3D{'sstable_size_in_mb': = '256', 'class': 'LeveledCompactionStrategy'}; >=20 > I still would have expected my original alter command to at least have = thrown an error. I assume I should open a bug for this? >=20 > alter table shard_user_lookup with compaction_strategy_options =3D = {'sstable_size_in_mb':256}; >=20 >=20 > From: Keith Wright > Reply-To: "user@cassandra.apache.org" > Date: Thursday, July 25, 2013 9:14 AM > To: "user@cassandra.apache.org" , Wei Zhu = > Subject: Re: sstable size change >=20 > Unfortunately the table in question is a CQL3 table so cassandra-cli = will not output its describe: >=20 > WARNING: CQL3 tables are intentionally omitted from 'describe' output. > See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details. >=20 > However, I did figure out that apparently I was not setting the change = properly. I was using the following alter via CQL3: >=20 > alter table shard_user_lookup with compaction_strategy_options =3D = {'sstable_size_in_mb':256}; >=20 > But when I did a describe, I did not see the change. =20 >=20 > CREATE TABLE shard_user_lookup ( > shard_user_id int, > shard text, > creation_time timestamp, > status int, > user_id timeuuid, > PRIMARY KEY (shard_user_id, shard) > ) WITH > bloom_filter_fp_chance=3D0.100000 AND > caching=3D'KEYS_ONLY' AND > comment=3D'' AND > dclocal_read_repair_chance=3D0.000000 AND > gc_grace_seconds=3D86400 AND > read_repair_chance=3D0.100000 AND > replicate_on_write=3D'true' AND > populate_io_cache_on_flush=3D'false' AND > compaction=3D{'class': 'LeveledCompactionStrategy'} AND > compression=3D{'chunk_length_kb': '8', 'crc_check_chance': '0.1', = 'sstable_compression': 'LZ4Compressor'}; >=20 > If I then execute the following via CQL2, I do see the property. = Seems odd to me? >=20 > alter table shard_user_lookup with = compaction_strategy_options:sstable_size_in_mb=3D256; >=20 > CREATE TABLE shard_user_lookup ( > shard_user_id int, > shard text, > creation_time timestamp, > status int, > user_id timeuuid, > PRIMARY KEY (shard_user_id, shard) > ) WITH > bloom_filter_fp_chance=3D0.100000 AND > caching=3D'KEYS_ONLY' AND > comment=3D'' AND > dclocal_read_repair_chance=3D0.000000 AND > gc_grace_seconds=3D86400 AND > read_repair_chance=3D0.100000 AND > replicate_on_write=3D'true' AND > populate_io_cache_on_flush=3D'false' AND > compaction=3D{'sstable_size_in_mb': '256', 'class': = 'LeveledCompactionStrategy'} AND > compression=3D{'chunk_length_kb': '8', 'crc_check_chance': '0.1', = 'sstable_compression': 'LZ4Compressor'}; >=20 >=20 > From: Wei Zhu > Reply-To: "user@cassandra.apache.org" , Wei = Zhu > Date: Wednesday, July 24, 2013 8:49 PM > To: "user@cassandra.apache.org" > Subject: Re: sstable size change >=20 > what is output of show keyspaces from cassandra-cli, did you see the = new value? >=20 > Compaction Strategy: = org.apache.cassandra.db.compaction.LeveledCompactionStrategy > Compaction Strategy Options: > sstable_size_in_mb: XXX >=20 > From: Keith Wright > To: "user@cassandra.apache.org" =20 > Sent: Wednesday, July 24, 2013 3:44 PM > Subject: Re: sstable size change >=20 > Hi all, >=20 > This morning I increased the SSTable size for one of my LCS via an = alter command and saw at least one compaction run (I did not trigger a = compaction via nodetool nor upgrades stables nor removing the .json = file). But so far my data sizes appear the same at the default 5 MB = (see below for output of ls =96Sal as well as relevant portion of = cfstats). Is this expected? I was hoping to see at least one file at = the new 256 MB size I set. >=20 > Thanks >=20 > SSTable count: 4965 > SSTables in each level: [0, 10, 112/100, 1027/1000, 3816, 0, 0, 0] > Space used (live): 29062393142 > Space used (total): 29140547702 > Number of Keys (estimate): 195103104 > Memtable Columns Count: 441483 > Memtable Data Size: 205486218 > Memtable Switch Count: 243 > Read Count: 154226729 >=20 > -rw-rw-r-- 1 cassandra cassandra 5247564 Jul 18 01:33 = users-shard_user_lookup-ib-97153-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247454 Jul 23 02:59 = users-shard_user_lookup-ib-109063-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247421 Jul 20 14:58 = users-shard_user_lookup-ib-103127-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247415 Jul 17 13:56 = users-shard_user_lookup-ib-95761-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247379 Jul 21 02:44 = users-shard_user_lookup-ib-104718-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247346 Jul 21 21:54 = users-shard_user_lookup-ib-106280-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247242 Jul 3 19:41 = users-shard_user_lookup-ib-66049-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247235 Jul 21 02:44 = users-shard_user_lookup-ib-104737-Data.db > -rw-rw-r-- 1 cassandra cassandra 5247233 Jul 20 14:58 = users-shard_user_lookup-ib-103169-Data.db >=20 >=20 > From: sankalp kohli > Reply-To: "user@cassandra.apache.org" > Date: Tuesday, July 23, 2013 3:04 PM > To: "user@cassandra.apache.org" > Subject: Re: sstable size change >=20 > "Will Cassandra force any newly compacted files to my new setting as = compactions are naturally triggered" > Yes. Let it compact and increase in size.=20 >=20 >=20 > On Tue, Jul 23, 2013 at 9:38 AM, Robert Coli = wrote: >> On Tue, Jul 23, 2013 at 6:48 AM, Keith Wright = wrote: >>> Can you elaborate on what you mean by "let it take its own course = organically"? Will Cassandra force any newly compacted files to my new = setting as compactions are naturally triggered? >>=20 >> You see, when two (or more!) SSTables love each other very much, they = sometimes decide they want to compact together.. >>=20 >> But seriously, "yes." If you force all existing SSTables to level 0, = it is as if you just flushed them all. Level compaction then does a = whole lot of compaction, using the active table size. >>=20 >> =3DRob >=20 >=20 >=20 --Apple-Mail=_9DA298A6-CEC3-4863-9769-AC168216A3A7 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252
I still would have = expected my original alter command to at least have thrown an error. =  I assume I should open a bug for this?
Sounds = like a good idea. 

Please describe the version = and query you first used. 

Thanks for = taking the time to update the = thread. 

Cheers

http://www.thelastpickle.com

On 27/07/2013, at 12:57 AM, Keith Wright <kwright@nanigans.com> = wrote:

FYI.  It = appears that the proper command in CQL3 is the = following:

alter table cookie_user_lookup with = compaction=3D{'sstable_size_in_mb': '256', 'class': = 'LeveledCompactionStrategy'};

I still would = have expected my original alter command to at least have thrown an = error.  I assume I should open a bug for = this?

alter table shard_user_lookup with = compaction_strategy_options =3D = {'sstable_size_in_mb':256};


From: = Keith Wright <kwright@nanigans.com>
Reply-To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;
Date: Thursday, July 25, = 2013 9:14 AM
To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;, Wei Zhu <wz1975@yahoo.com>
Subject: Re: sstable size = change

Unfortunately = the table in question is a CQL3 table so cassandra-cli will not output = its describe:

WARNING: CQL3 tables are = intentionally omitted from 'describe' output.

However, I did figure out that = apparently I was not setting the change properly.  I was using the = following alter via CQL3:

alter table = shard_user_lookup with compaction_strategy_options =3D = {'sstable_size_in_mb':256};

But when I did a = describe, I did not see the change. =   

CREATE TABLE = shard_user_lookup (
  shard_user_id int,
  = shard text,
  creation_time timestamp,
  = status int,
  user_id timeuuid,
  PRIMARY = KEY (shard_user_id, shard)
) WITH
  = bloom_filter_fp_chance=3D0.100000 AND
  = caching=3D'KEYS_ONLY' AND
  comment=3D'' = AND
  dclocal_read_repair_chance=3D0.000000 = AND
  gc_grace_seconds=3D86400 AND
  = read_repair_chance=3D0.100000 AND
  = replicate_on_write=3D'true' AND
  = populate_io_cache_on_flush=3D'false' AND
  = compaction=3D{'class': 'LeveledCompactionStrategy'} AND
  = compression=3D{'chunk_length_kb': '8', 'crc_check_chance': '0.1', = 'sstable_compression': = 'LZ4Compressor'};

If I then execute the = following via CQL2, I do see the property.  Seems odd to = me?

alter table shard_user_lookup with = compaction_strategy_options:sstable_size_in_mb=3D256;

=
CREATE TABLE shard_user_lookup (
  = shard_user_id int,
  shard text,
  = creation_time timestamp,
  status int,
  = user_id timeuuid,
  PRIMARY KEY (shard_user_id, = shard)
) WITH
  bloom_filter_fp_chance=3D0.100000= AND
  caching=3D'KEYS_ONLY' AND
  = comment=3D'' AND
  dclocal_read_repair_chance=3D0.000000 = AND
  gc_grace_seconds=3D86400 AND
  = read_repair_chance=3D0.100000 AND
  = replicate_on_write=3D'true' AND
  = populate_io_cache_on_flush=3D'false' AND
  = compaction=3D{'sstable_size_in_mb': '256', 'class': = 'LeveledCompactionStrategy'} AND
  = compression=3D{'chunk_length_kb': '8', 'crc_check_chance': '0.1', = 'sstable_compression': = 'LZ4Compressor'};


From: Wei = Zhu <wz1975@yahoo.com>
Reply-To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;, Wei Zhu <wz1975@yahoo.com>
Date: Wednesday, July 24, 2013 8:49 = PM
To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;
Subject: Re: sstable size = change

what is output of show keyspaces from = cassandra-cli, did you see the new value?
     =   Compaction Strategy Options:
        = sstable_size_in_mb: XXX

From: = Keith Wright <kwright@nanigans.com>
To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;
Sent: Wednesday, = July 24, 2013 3:44 PM
Subject: Re: sstable size change

Hi = all,

   This morning I increased the = SSTable size for one of my LCS via an alter command and saw at least one = compaction run (I did not trigger a compaction via nodetool nor upgrades = stables nor removing the .json file).  But so far my data sizes = appear the same at the default 5 MB (see below for output of ls =96Sal as well as = relevant portion of cfstats).   Is this expected?  I was = hoping to see at least one file at the new 256 MB size I = set.

Thanks

SSTable= count: 4965
SSTables in each level: [0, 10, = 112/100, 1027/1000, 3816, 0, 0, 0]
Space used (live): = 29062393142
Space used (total): = 29140547702
Number of Keys (estimate): = 195103104
Memtable Columns Count: = 441483
Memtable Data Size: = 205486218
Memtable Switch Count: = 243
Read Count: = 154226729

-rw-rw-r--  1 = cassandra cassandra 5247564 Jul 18 01:33 = users-shard_user_lookup-ib-97153-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247454 Jul 23 02:59 = users-shard_user_lookup-ib-109063-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247421 Jul 20 14:58 = users-shard_user_lookup-ib-103127-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247415 Jul 17 13:56 = users-shard_user_lookup-ib-95761-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247379 Jul 21 02:44 = users-shard_user_lookup-ib-104718-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247346 Jul 21 21:54 = users-shard_user_lookup-ib-106280-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247242 Jul  3 19:41 = users-shard_user_lookup-ib-66049-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247235 Jul 21 02:44 = users-shard_user_lookup-ib-104737-Data.db
-rw-rw-r--  1 = cassandra cassandra 5247233 Jul 20 14:58 = users-shard_user_lookup-ib-103169-Data.db

<= br>
From: sankalp kohli <kohlisankalp@gmail.com>
<= span style=3D"font-weight:bold;">Reply-To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;
Date: Tuesday, July 23, = 2013 3:04 PM
To: "user@cassandra.apache.org" = <user@cassandra.apache.org>= ;
Subject: Re: sstable size = change

"Will = Cassandra force any newly compacted files to my new setting as = compactions are naturally triggered"
Yes. Let it compact and increase in = size. 


On Tue, Jul 23, 2013 at 9:38 AM, = Robert Coli <rcoli@eventbrite.com> = wrote:
On Tue, Jul 23, 2013 at 6:48 = AM, Keith Wright <kwright@nanigans.com> = wrote:
Can you elaborate on what you mean = by "let it take its own course organically"?  Will Cassandra force = any newly compacted files to my new setting as compactions are naturally = triggered?

You see, when two = (or more!) SSTables love each other very much, they sometimes decide = they want to compact together..

But seriously, = "yes." If you force all existing SSTables to level 0, it is as if you = just flushed them all. Level compaction then does a whole lot of = compaction, using the active table = size.

=3DRob
=



<= /span>
= --Apple-Mail=_9DA298A6-CEC3-4863-9769-AC168216A3A7--