Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 98E9A9ED2 for ; Mon, 5 Mar 2012 17:29:57 +0000 (UTC) Received: (qmail 26752 invoked by uid 500); 5 Mar 2012 17:29:55 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 26705 invoked by uid 500); 5 Mar 2012 17:29:55 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 26694 invoked by uid 99); 5 Mar 2012 17:29:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Mar 2012 17:29:55 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a57.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Mar 2012 17:29:47 +0000 Received: from homiemail-a57.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a57.g.dreamhost.com (Postfix) with ESMTP id 03C1820806B for ; Mon, 5 Mar 2012 09:29:24 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=f88/P10Mkr jsKCqrvr9UtPoEhkvFcr0U+D4tRAOY5zg5cAvYzqlegHOCQG6FY51FXc85nbDkfC tA7Mk1BMXF5dVksb+FP5XD30zeOd6KzxftEvs7AXWGKYefrXZmgLzEAM674I5RTi cpE7DXkunBXRFtMFA6y4xO+gZ3UtEeXnI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=HNzGXtZQ5oRskyJH rDszsTzVClQ=; b=ZiiLI+96OHXayXCvcfGXhvSmvfRSu3IuLCpmAW2LjDxgsgBY jZdby5bkbkJ3MYVnIk2lXsbdjV5cWC0V3Nkv8M8T3MhE/CSbGXJ7KfowUd4ZWnSh UrQ80wjy76SBK81bXNsCc1T2u9td3Fk98DObNeMujej3NtLt974blvhJahg= Received: from [172.16.1.3] (125-236-193-159.adsl.xtra.co.nz [125.236.193.159]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a57.g.dreamhost.com (Postfix) with ESMTPSA id 640F4208060 for ; Mon, 5 Mar 2012 09:29:24 -0800 (PST) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: multipart/alternative; boundary="Apple-Mail=_9557C342-CC13-4D2F-A3F0-6601BC14BAC5" Subject: Re: Issue with nodetool clearsnapshot Date: Tue, 6 Mar 2012 06:29:20 +1300 In-Reply-To: To: user@cassandra.apache.org References: Message-Id: <7376F52D-1C4C-4BF0-95A1-528BF3C3D85A@thelastpickle.com> X-Mailer: Apple Mail (2.1257) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_9557C342-CC13-4D2F-A3F0-6601BC14BAC5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 > It seems that instead of removing the snapshot, clearsnapshot moved = the data files from the snapshot directory to the parent directory and = the size of the data for that keyspace has doubled. That is not possible, there is only code there to delete a files in the = snapshot.=20 Note that in the snapshot are hard links to the files in the data dir. = Deleting / clearing the snapshot will not delete the files from the data = dir if they are still in use.=20 > Many of the files are looking like duplicates. >=20 > in Keyspace1 directory > 156987786084 Jan 21 03:18 Standard1-g-7317-Data.db > 156987786084 Mar 4 01:33 Standard1-g-8850-Data.db Under 0.8.x files are not immediately deleted. Did the data directory = contain zero size -Compacted files with the same number ? =20 Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 5/03/2012, at 11:50 PM, B R wrote: > Version 0.8.9 >=20 > We run a 2 node cluster with RF=3D2. We ran a scrub and after that ran = the clearsnapshot to remove the backup snapshot created by scrub. It = seems that instead of removing the snapshot, clearsnapshot moved the = data files from the snapshot directory to the parent directory and the = size of the data for that keyspace has doubled. Many of the files are = looking like duplicates. >=20 > in Keyspace1 directory > 156987786084 Jan 21 03:18 Standard1-g-7317-Data.db > 156987786084 Mar 4 01:33 Standard1-g-8850-Data.db > 118211555728 Jan 31 12:50 Standard1-g-7968-Data.db > 118211555728 Mar 3 22:58 Standard1-g-8840-Data.db > 116902342895 Feb 25 02:04 Standard1-g-8832-Data.db > 116902342895 Mar 3 22:10 Standard1-g-8836-Data.db > 93788425710 Feb 21 04:20 Standard1-g-8791-Data.db > 93788425710 Mar 4 00:29 Standard1-g-8845-Data.db > ..... >=20 > Even though the nodetool ring command shows the correct data size for = the node, the du -sh on the keyspace directory gives double the size. >=20 > Can you guide us to proceed from this situation ? >=20 > Thanks. --Apple-Mail=_9557C342-CC13-4D2F-A3F0-6601BC14BAC5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
It= seems that instead of removing the snapshot, clearsnapshot moved the = data files from the snapshot directory to the parent directory and the = size of the data for that keyspace has doubled.
That is not = possible, there is only code there to delete a files in the = snapshot. 

Note that in the snapshot are = hard links to the files in the data dir. Deleting / clearing the = snapshot will not delete the files from the data dir if they are still = in use. 

 Many of the files are looking like = duplicates.

in Keyspace1 directory
156987786084 Jan 21 03:18 = Standard1-g-7317-Data.db
156987786084 Mar  4 01:33 = Standard1-g-8850-Data.db
Under 0.8.x files are not = immediately deleted. Did the data directory contain zero size -Compacted = files with the same number = ?
  
Cheers

http://www.thelastpickle.com

On 5/03/2012, at 11:50 PM, B R wrote:

Version = 0.8.9

We run a 2 node cluster with RF=3D2. We ran a scrub and = after that ran the clearsnapshot to remove the backup snapshot created = by scrub. It seems that instead of removing the snapshot, clearsnapshot = moved the data files from the snapshot directory to the parent directory = and the size of the data for that keyspace has doubled. Many of the = files are looking like duplicates.

in Keyspace1 directory
156987786084 Jan 21 03:18 = Standard1-g-7317-Data.db
156987786084 Mar  4 01:33 = Standard1-g-8850-Data.db
118211555728 Jan 31 12:50 = Standard1-g-7968-Data.db
118211555728 Mar  3 22:58 = Standard1-g-8840-Data.db
116902342895 Feb 25 02:04 Standard1-g-8832-Data.db
116902342895 = Mar  3 22:10 Standard1-g-8836-Data.db
93788425710 Feb 21 04:20 = Standard1-g-8791-Data.db
93788425710 Mar  4 00:29 = Standard1-g-8845-Data.db
.....

Even though the nodetool ring command shows the correct data size = for the node, the du -sh on the keyspace directory gives double the = size.

Can you guide us to proceed from this situation = ?

Thanks.

= --Apple-Mail=_9557C342-CC13-4D2F-A3F0-6601BC14BAC5--