Return-Path: 
 <user-return-34427-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 1274110059
	for <apmail-cassandra-user-archive@www.apache.org>;
 Mon,  3 Jun 2013 19:14:39 +0000 (UTC)
Received: (qmail 80881 invoked by uid 500); 3 Jun 2013 19:14:36 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 80834 invoked by uid 500); 3 Jun 2013 19:14:36 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 80826 invoked by uid 99); 3 Jun 2013 19:14:36 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 19:14:36 +0000
X-ASF-Spam-Status: No, hits=-0.7 required=5.0
	tests=RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of rcoli@eventbrite.com designates
 209.85.128.46 as permitted sender)
Received: from [209.85.128.46] (HELO mail-qe0-f46.google.com) (209.85.128.46)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 19:14:30 +0000
Received: by mail-qe0-f46.google.com with SMTP id w7so1785654qeb.5
        for <user@cassandra.apache.org>; Mon, 03 Jun 2013 12:14:08 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:x-gm-message-state;
        bh=Ot8hPLvxdRufsmoHM4/Khozj5NIQEeIvFTTTqPSKn+w=;
        b=JtZzd3YeXgEJ3Jvw/lUJZzHGjFQo02MyszwI4Aq1AHTO2MhnzTdBRGHDD20n3CqxbR
         6kPB5t28/fUIwEYvGbV27BK2GjRyFbHgGroGj8DPCngUfkus/e92Uj9dR2+zs/HE2shw
         RozYa4VChFFWFfI2RKUQBBkTDNNu3mmjlDe1QaAkuPdM2oKwvGMNUwkbsCBUSAZGRvOL
         6oXGizRnMlKq2CYXVVkTOmDic4xejGa1lL0vKiXehT3YN/G7nnLE3WGxif27Npfk1OL4
         8S01JU7G8FHE3DqDslrA1mxkssuz/NZ3NHA+QtIr9ddCh3HpNHih2bqs8ZKclPuuLd0L
         GmgA==
MIME-Version: 1.0
X-Received: by 10.224.39.77 with SMTP id f13mr20233251qae.96.1370286848623;
 Mon, 03 Jun 2013 12:14:08 -0700 (PDT)
Received: by 10.49.5.135 with HTTP; Mon, 3 Jun 2013 12:14:08 -0700 (PDT)
In-Reply-To: <alpine.DEB.2.00.1306031453380.9651@computableinsights.com>
References: <alpine.DEB.2.00.1306031453380.9651@computableinsights.com>
Date: Mon, 3 Jun 2013 12:14:08 -0700
Message-ID: 
 <CAEDUwd2_Jx1+F94w3-WdQgz2M1w1jOq31O4a3yLyMSPiMAOz+Q@mail.gmail.com>
Subject: Re: unable to delete
From: Robert Coli <rcoli@eventbrite.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
X-Gm-Message-State: 
 ALoCoQmzfGXkdoKLps/WPl4aYVkXqhs8jCu2+eMnFhFIFTC5Ewy6oB3ZSCHdpymv8ESXLOm1wRJl
X-Virus-Checked: Checked by ClamAV on apache.org

On Mon, Jun 3, 2013 at 11:57 AM, John R. Frank <jrf@mit.edu> wrote:
> Is it considered normal for cassandra to experience this error:
>
> ERROR [NonPeriodicTasks:1] 2013-06-03 18:17:05,374 SSTableDeletingTask.java
> (line 72) Unable to delete
> /raid0/cassandra/data/<KEYSPACE>/<CF>/<KEYSPACE>-<CF>-ic-19-Data.db (it will
> be removed on server restart; we'll also retry after GC)

cassandra//src/java/org/apache/cassandra/io/sstable/SSTableDeletingTask.java
"
       File datafile = new File(desc.filenameFor(Component.DATA));
        if (!datafile.delete())
        {
            logger.error("Unable to delete " + datafile + " (it will
be removed on server restart; we'll also retry after GC)");
            failedTasks.add(this);
            return;
        }
"

There are contexts where it is appropriate for Cassandra to be unable
to delete a file using io.File.delete.

"
// Deleting sstables is tricky because the mmapping might not have
been finalized yet,
// and delete will fail (on Windows) until it is (we only force the
unmapping on SUN VMs).
// Additionally, we need to make sure to delete the data file first,
so on restart the others
// will be recognized as GCable.
"

Do the files get deleted on GC/server restart?

> This is on the DataStax EC2 AMI in a two-node cluster.  After deleting 1,000
> rows from a CF with 20,000 rows, the DB becomes slow, and I'm trying to
> figure out why.  Could this error message be pointing at a proximate cause?

Almost certainly not. By the time that a sstable file is subject to
deletion, it should no longer be "live". When it is no longer "live"
it is not in the read path.

You can verify this by using nodetool getsstables on a given key.

What operation are you trying to do when the "DB becomes slow"?

=Rob