Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9C2D5C685 for ; Sun, 20 May 2012 23:56:07 +0000 (UTC) Received: (qmail 29320 invoked by uid 500); 20 May 2012 23:56:05 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 29291 invoked by uid 500); 20 May 2012 23:56:05 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 29282 invoked by uid 99); 20 May 2012 23:56:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 20 May 2012 23:56:05 +0000 X-ASF-Spam-Status: No, hits=3.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a48.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 20 May 2012 23:55:57 +0000 Received: from homiemail-a48.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a48.g.dreamhost.com (Postfix) with ESMTP id 86FFB4F8058 for ; Sun, 20 May 2012 16:55:35 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=cu63iEuNcw L8OFCtPOi/xKit1tUCsrvpuooK6sAYzQ5bxlzIy1CG5/N5U3JiTqBP0RqBW08B5G ckubGwvLW0+h+XhZw0AE4J1x7ap1DhJYvhVVszpGgWI+vTbbxLcv+fJSD1H/bmXs I/FM2OsGWPw32u5fh0SWyDbM/95dKAYfw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=Eaj4ELahPja53cqK nF3ekJ+Ah94=; b=l8hlxMcdby67fxCJEKqBM98i63AzD3DJVTXXE93Y1/XpLATY 2/7vwXOK37qGD6e1mv0agr/FVVxxMYGiVTb3LOgdqdUhfT1aTItQVqtlrzZjpxe/ /sL5qQ+Vyy10qNGa2uPd1vBH7sczJO6er3We2vBi9H2jLn71hJ+6d0mg9SE= Received: from [172.16.1.4] (unknown [203.86.207.101]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a48.g.dreamhost.com (Postfix) with ESMTPSA id 7B5B04F8055 for ; Sun, 20 May 2012 16:55:34 -0700 (PDT) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: multipart/alternative; boundary="Apple-Mail=_CE385192-50A8-40E3-9DF5-103E786F52CD" Subject: Re: Safely Disabling Compaction Date: Mon, 21 May 2012 11:55:24 +1200 In-Reply-To: To: user@cassandra.apache.org References: <1337204626125-7562777.post@n2.nabble.com> Message-Id: X-Mailer: Apple Mail (2.1257) --Apple-Mail=_CE385192-50A8-40E3-9DF5-103E786F52CD Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii The docs for the CLI say: - min_compaction_threshold: The minimum number of SSTables needed to start a minor compaction. Default is 4, setting to 0 disables minor compactions. If you are getting an error using 0 can you please raise a ticket on = https://issues.apache.org/jira/browse/CASSANDRA (Also you can also set the compaction threshold on a single node via = nodetool. ) Thanks ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 18/05/2012, at 3:09 PM, Vijay wrote: > I would rather set the Keyspace setting min_compaction_threshold and = max_compaction_threshold to be a higher number and once i am ready i = will put the value back... This way i dont need to restart.=20 > Having said that why not set the compaction throughput to 1 (low = enough to not have contention) and complete the stream? >=20 > Regards, > >=20 >=20 >=20 > On Wed, May 16, 2012 at 2:43 PM, sj.climber = wrote: > Hi, >=20 > In an effort to minimize IO contention, I'd like to disable = compactions > while I'm streaming SSTables to the cluster. When done streaming, I = intend > on forcing a major compaction through nodetool. >=20 > Elsewhere in the forums, various folks suggest setting > max_compaction_threshold =3D 0 to disable compaction. While this = works > sometimes (via 'update column family with > max_compaction_threshold=3D0'), I've observed a number of serious = issues with > this approach: >=20 > 1) You can't create a column family with max_compaction_threshold =3D = 0. The > CLI reports that min_compaction_threshold must have a value >=3D 2, = and > max_compaction_threshold can't be lower than it. Worse yet, trying to > create a column family with max_compaction_threshold =3D 0 gets the = cluster > into a Schema Disagreement Exception (since the node on which you = issue the > migration command fails with a fatal error). >=20 > 2) Cassandra will allow me to update an existing column family with > max_compaction_threshold =3D 0. But if I restart the node, it will = crash on > startup. > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > = sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 39) > at > = sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > = org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)= > Caused by: java.lang.RuntimeException: > java.lang.reflect.InvocationTargetException > ... > = org.apache.cassandra.config.CFMetaData.createCompactionStrategyInstance(CF= MetaData.java:839) > ... 14 more > Caused by: java.lang.RuntimeException: The max_compaction_threshold = cannot > be smaller than the min. > at > = org.apache.cassandra.db.ColumnFamilyStore.setMaximumCompactionThreshold(Co= lumnFamilyStore.java:1740) > at org.apache. >=20 >=20 > Is there another solution for more safely enabling/disabling = compaction? >=20 > Thanks! >=20 > -- > View this message in context: = http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Safely-Di= sabling-Compaction-tp7562777.html > Sent from the cassandra-user@incubator.apache.org mailing list archive = at Nabble.com. >=20 --Apple-Mail=_CE385192-50A8-40E3-9DF5-103E786F52CD Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii https://issues.ap= ache.org/jira/browse/CASSANDRA

(Also you = can also set the compaction threshold on a single node via nodetool. = )

Thanks
=
http://www.thelastpickle.com

On 18/05/2012, at 3:09 PM, Vijay wrote:

I would = rather set the Keyspace setting min_compaction_threshold = and max_compaction_threshold to be a higher number and once i am = ready i will put the value back... This way i dont need to = restart. 
Having said that why not set the compaction = throughput to 1 (low enough to not have contention) and complete the = stream?

Regards,
</VJ>



On Wed, May 16, 2012 at 2:43 PM, = sj.climber <sj.climber@gmail.com> = wrote:
Hi,

In an effort to minimize IO contention, I'd like to disable = compactions
while I'm streaming SSTables to the cluster.  When done streaming, = I intend
on forcing a major compaction through nodetool.

Elsewhere in the forums, various folks suggest setting
max_compaction_threshold =3D 0 to disable compaction.  While this = works
sometimes (via 'update column family <family> with
max_compaction_threshold=3D0'), I've observed a number of serious issues = with
this approach:

1) You can't create a column family with max_compaction_threshold =3D 0. =  The
CLI reports that min_compaction_threshold must have a value >=3D 2, = and
max_compaction_threshold can't be lower than it.  Worse yet, trying = to
create a column family with max_compaction_threshold =3D 0 gets the = cluster
into a Schema Disagreement Exception (since the node on which you issue = the
migration command fails with a fatal error).

2) Cassandra will allow me to update an existing column family with
max_compaction_threshold =3D 0.  But if I restart the node, it will = crash on
startup.
java.lang.reflect.InvocationTargetException
       at = sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at
= sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 39)
       at
= sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:25)
       at = java.lang.reflect.Method.invoke(Method.java:597)
       at
= org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)=
Caused by: java.lang.RuntimeException:
java.lang.reflect.InvocationTargetException
...
= org.apache.cassandra.config.CFMetaData.createCompactionStrategyInstance(CF= MetaData.java:839)
       ... 14 more
Caused by: java.lang.RuntimeException: The max_compaction_threshold = cannot
be smaller than the min.
       at
= org.apache.cassandra.db.ColumnFamilyStore.setMaximumCompactionThreshold(Co= lumnFamilyStore.java:1740)
       at org.apache.


Is there another solution for more safely enabling/disabling = compaction?

Thanks!

--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.na= bble.com/Safely-Disabling-Compaction-tp7562777.html
Sent from the cassandra-user@incubat= or.apache.org mailing list archive at Nabble.com.


= --Apple-Mail=_CE385192-50A8-40E3-9DF5-103E786F52CD--