From user-return-35069-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Fri Jul 5 07:10:54 2013 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E27DD10925 for ; Fri, 5 Jul 2013 07:10:54 +0000 (UTC) Received: (qmail 83262 invoked by uid 500); 5 Jul 2013 07:10:52 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 83057 invoked by uid 500); 5 Jul 2013 07:10:52 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 83049 invoked by uid 99); 5 Jul 2013 07:10:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jul 2013 07:10:51 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [88.198.227.104] (HELO team.enercast.de) (88.198.227.104) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jul 2013 07:10:45 +0000 Received: from [172.17.2.54] (alpha.enercast.de [217.86.247.221]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by team.enercast.de (Postfix) with ESMTPSA id 85235B400A3 for ; Fri, 5 Jul 2013 07:10:24 +0000 (UTC) Message-ID: <51D67160.8010508@enercast.de> Date: Fri, 05 Jul 2013 09:10:24 +0200 From: Jan Kesten User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130116 Icedove/10.0.12 MIME-Version: 1.0 To: user@cassandra.apache.org Subject: Re: CorruptBlockException - recover? References: <93fa6a475304c9d60e1f9a321c76a2b7@team.enercast.de> In-Reply-To: Content-Type: multipart/alternative; boundary="------------040807070808010705030603" X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------040807070808010705030603 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Hi, i tried to scrub the keyspace - but with no success either, the process threw an exception when hitting the corrupt block and stopped then. I will rebootstrap the node :-) Thanks anyways, Jan On 03.07.2013 19:10, Glenn Thompson wrote: > For what its worth. I did this when I had this problem. It didn't > work out for me. Perhaps I did something wrong. > > > On Wed, Jul 3, 2013 at 11:06 AM, Robert Coli > wrote: > > On Wed, Jul 3, 2013 at 7:04 AM, ifjke > wrote: > > I found that one of my cassandra nodes died recently (machine > hangs). I restarted the node an run a nodetool repair, while > running it has thrown a org.apache.cassandra.io > .compress.CorruptBlockException. > Is there any way to recover from this? Or would it be best to > delete the nodes contents and bootstrap it again? > > > If you "scrub" this SSTable (either with the online or offline > version of "scrub") it will remove the corrupt data and re-write > the rest of the SSTable which isn't corrupt into a new SSTable. > That is probably safer for your data than deleting the entire set > of data on this replica. When that's done, restart the repair. > > =Rob > > -- Jan Kesten, mailto:j.kesten@enercast.de Tel.: +49 561/4739664-0 FAX: -9 enercast GmbH Friedrich-Ebert-Str. 104 D-34119 Kassel HRB15471 http://www.enercast.de Online-Prognosen für erneuerbare Energien Geschäftsführung: Dipl. Ing. Thomas Landgraf, Bernd Kratz Diese E-Mail und etwaige Anhänge können vertrauliche und/oder rechtlich geschützte Informationen enthalten. Falls Sie nicht der angegebene Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde, benachrichtigen Sie uns bitte sofort durch Antwort-E-Mail und löschen Sie diese E-Mail nebst etwaigen Anlagen von Ihrem System. Ebenso dürfen Sie diese E-Mail oder ihre Anlagen nicht kopieren oder an Dritte weitergeben. Vielen Dank. This e-mail and any attachment may contain confidential and/or privileged information. If you are not the named addressee or if this transmission has been addressed to you in error, please notify us immediately by reply e-mail and then delete this e-mail and any attachment from your system. Please understand that you must not copy this e-mail or any attachment or disclose the contents to any other person. Thank you for your cooperation. --------------040807070808010705030603 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi,

i tried to scrub the keyspace - but with no success either, the process threw an exception when hitting the corrupt block and stopped then. I will rebootstrap the node :-)

Thanks anyways,
Jan

On 03.07.2013 19:10, Glenn Thompson wrote:
For what its worth.  I did this when I had this problem.  It didn't work out for me.  Perhaps I did something wrong.


On Wed, Jul 3, 2013 at 11:06 AM, Robert Coli <rcoli@eventbrite.com> wrote:
On Wed, Jul 3, 2013 at 7:04 AM, ifjke <j.kesten@enercast.de> wrote:
I found that one of my cassandra nodes died recently (machine hangs). I restarted the node an run a nodetool repair, while running it has thrown a org.apache.cassandra.io.compress.CorruptBlockException. Is there any way to recover from this? Or would it be best to delete the nodes contents and bootstrap it again?

If you "scrub" this SSTable (either with the online or offline version of "scrub") it will remove the corrupt data and re-write the rest of the SSTable which isn't corrupt into a new SSTable. That is probably safer for your data than deleting the entire set of data on this replica. When that's done, restart the repair.

=Rob



-- 
Jan Kesten, mailto:j.kesten@enercast.de
Tel.: +49 561/4739664-0 FAX: -9
enercast GmbH Friedrich-Ebert-Str. 104 D-34119 Kassel       HRB15471
http://www.enercast.de Online-Prognosen für erneuerbare Energien
Geschäftsführung: Dipl. Ing. Thomas Landgraf, Bernd Kratz

Diese E-Mail und etwaige Anhänge können vertrauliche und/oder rechtlich geschützte Informationen enthalten. Falls Sie nicht der angegebene Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde, benachrichtigen Sie uns bitte sofort durch Antwort-E-Mail und löschen Sie diese E-Mail nebst etwaigen Anlagen von Ihrem System. Ebenso dürfen Sie diese E-Mail oder ihre Anlagen nicht kopieren oder an Dritte weitergeben. Vielen Dank.

This e-mail and any attachment may contain confidential and/or privileged information. If you are not the named addressee or if this transmission has been addressed to you in error, please notify us immediately by reply e-mail and then delete this e-mail and any attachment from your system. Please understand that you must not copy this e-mail or any attachment or disclose the contents to any other person. Thank you for your cooperation. 
--------------040807070808010705030603--