From user-return-30604-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Thu Dec 13 11:51:24 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8A1C9D6DB for ; Thu, 13 Dec 2012 11:51:24 +0000 (UTC) Received: (qmail 51735 invoked by uid 500); 13 Dec 2012 11:51:22 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 51093 invoked by uid 500); 13 Dec 2012 11:51:20 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 51037 invoked by uid 500); 13 Dec 2012 11:51:19 -0000 Delivered-To: apmail-incubator-cassandra-user@incubator.apache.org Received: (qmail 51030 invoked by uid 99); 13 Dec 2012 11:51:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Dec 2012 11:51:19 +0000 X-ASF-Spam-Status: No, hits=2.0 required=5.0 tests=SPF_NEUTRAL,URI_HEX X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: 216.139.250.139 is neither permitted nor denied by domain of solf.lists@gmail.com) Received: from [216.139.250.139] (HELO joe.nabble.com) (216.139.250.139) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Dec 2012 11:51:14 +0000 Received: from jim.nabble.com ([192.168.236.80]) by joe.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1Tj7JV-0008T5-EM for cassandra-user@incubator.apache.org; Thu, 13 Dec 2012 03:50:53 -0800 Date: Thu, 13 Dec 2012 03:50:53 -0800 (PST) From: Sergey Olefir To: cassandra-user@incubator.apache.org Message-ID: <1355399453369-7584264.post@n2.nabble.com> In-Reply-To: <7478EB05-FF35-4A7E-A037-52A961CFFFD8@thelastpickle.com> References: <1355234434695-7584206.post@n2.nabble.com> <3EBEA264-9711-45E2-A1C9-715A7D592FF7@thelastpickle.com> <1355268235485-7584232.post@n2.nabble.com> <1355345296802-7584254.post@n2.nabble.com> <1355348137924-7584256.post@n2.nabble.com> <7478EB05-FF35-4A7E-A037-52A961CFFFD8@thelastpickle.com> Subject: Re: Multiple Data Center shows very uneven load MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org I'll try nodetool drain, thanks. But more generally -- are you basically saying that I should not worry about these things? Data will not keep accumulating indefinitely in production and it'll not affect performance negatively (despite vast differences in node load)? Best regards, Sergey aaron morton wrote > try nodetool drain. It will flush everything to disk and the commit log > will be truncated. > > HH can be ignored. If you really want them gone they can be purged using > the JMX interface, or you can stop the node and delete the sstables. > > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Developer > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 13/12/2012, at 10:35 AM, Sergey Olefir < > solf.lists@ > > wrote: > >> Nick Bailey-2 wrote >>> Dropping a keyspace causes a snapshot to be taken of the keyspace before >>> it >>> is removed from the schema. So it won't actually delete any data. You >>> can >>> manually delete the data from /var/lib/cassandra/ >>> > >>> /<cf[s]>/snapshots >> >> Indeed, it looks like snapshot is on the file system. However it looks >> like >> it is not the only thing by a long shot, i.e.: >> cassa1-1:/var/log/cassandra# du -k /spool1/cassandra/data/1.1/ >> 375372 >> /spool1/cassandra/data/1.1/rainmanLoadTestKeyspace/marquisColumnFamily/snapshots/1355222054452-marquisColumnFamily >> 375376 >> /spool1/cassandra/data/1.1/rainmanLoadTestKeyspace/marquisColumnFamily/snapshots >> 375380 >> /spool1/cassandra/data/1.1/rainmanLoadTestKeyspace/marquisColumnFamily >> 375384 /spool1/cassandra/data/1.1/rainmanLoadTestKeyspace >> 4 /spool1/cassandra/data/1.1/system/Versions >> 52 /spool1/cassandra/data/1.1/system/schema_columns >> 4 /spool1/cassandra/data/1.1/system/Schema >> 28 /spool1/cassandra/data/1.1/system/NodeIdInfo >> 4 /spool1/cassandra/data/1.1/system/Migrations >> 28 /spool1/cassandra/data/1.1/system/schema_keyspaces >> 28 /spool1/cassandra/data/1.1/system/schema_columnfamilies >> 786348 /spool1/cassandra/data/1.1/system/HintsColumnFamily >> 52 /spool1/cassandra/data/1.1/system/LocationInfo >> 4 /spool1/cassandra/data/1.1/system/IndexInfo >> 786556 /spool1/cassandra/data/1.1/system >> 1161944 /spool1/cassandra/data/1.1/ >> >> >> And also 700+MB in the commitlog. Neither of which seemed to 'go away' on >> its own when idle or even after running nodetool repair/cleanup and even >> dropping keyspace. >> >> I suppose these hints and commitlog may be the reason behind huge >> difference >> in load on nodes -- but why does it happen and more importantly is it >> harmful? Will it keep accumulating? >> >> >> >> -- >> View this message in context: >> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Multiple-Data-Center-shows-very-uneven-load-tp7584197p7584256.html >> Sent from the > cassandra-user@.apache > mailing list archive at Nabble.com. -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Multiple-Data-Center-shows-very-uneven-load-tp7584197p7584264.html Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.