Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 322FA1028B for ; Mon, 5 Aug 2013 14:45:55 +0000 (UTC) Received: (qmail 14415 invoked by uid 500); 5 Aug 2013 14:45:52 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 13935 invoked by uid 500); 5 Aug 2013 14:45:51 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 13623 invoked by uid 99); 5 Aug 2013 14:45:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Aug 2013 14:45:42 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of peichieh@gmail.com designates 209.85.219.44 as permitted sender) Received: from [209.85.219.44] (HELO mail-oa0-f44.google.com) (209.85.219.44) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Aug 2013 14:45:19 +0000 Received: by mail-oa0-f44.google.com with SMTP id l20so6522958oag.3 for ; Mon, 05 Aug 2013 07:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=RpqxdV3Ou+7RgIbtkG9xTqHT81yd6ZKQYXyrZervDXw=; b=uyi+ja1B27ip4kF/WsU/mWEv864SY1f87qKUcRskw0rqT8JY9vpbQA18CtlirO9HcC 7r/3+UnU6Tbsyv5tto2xDxEvUP3FwBAXH4GKEel+e0/QnzTCPq0txNEf/8Wu1OskYKEo i9XMoG85FoHh+OmOmXZtREiwlUAbT1eX3OvLXw8jqcEaLFE36PZr6zQTySZZHtdoaxBi qFk+j4w5cz+ixsMiKs3qK+sFxAfG5T+vOBOruoB3l3W/dmeGTE5zl2Rcvc2wVHPmp3DJ m0q019kPlR4Iev6IukjzshSJQpWwT7hXPJ2g6Ivlvxu4iTBRaSMSqNf9leVuo1AECNs7 FAgQ== MIME-Version: 1.0 X-Received: by 10.42.228.1 with SMTP id jc1mr1658548icb.92.1375713897984; Mon, 05 Aug 2013 07:44:57 -0700 (PDT) Received: by 10.64.136.6 with HTTP; Mon, 5 Aug 2013 07:44:57 -0700 (PDT) In-Reply-To: References: Date: Mon, 5 Aug 2013 22:44:57 +0800 Message-ID: Subject: Re: org.apache.cassandra.io.sstable.CorruptSSTableException From: Jason Wee To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a1133d7b833ba7204e3345aa9 X-Virus-Checked: Checked by ClamAV on apache.org --001a1133d7b833ba7204e3345aa9 Content-Type: text/plain; charset=ISO-8859-1 you can try nodetool scrub. if it does not work, try repair then cleanup. had this issue a few weeks back but our version is 1.0.x On Mon, Aug 5, 2013 at 8:12 AM, Keith Wright wrote: > Re-sending hoping to get some help. Any ideas would be much appreciated! > > From: Keith Wright > Date: Friday, August 2, 2013 3:01 PM > To: "user@cassandra.apache.org" > Subject: org.apache.cassandra.io.sstable.CorruptSSTableException > > Hi all, > > We just added a node to our cluster (1.2.4 Vnodes) and they appear to > be running well exception I see that the new node is not making any > progress compacting one of the CF. The exception below is generated. My > assumption is that the only way to handle this is to stop the node, delete > the file in question, restart, and run repair. > > Thoughts? > > org.apache.cassandra.io.sstable.CorruptSSTableException: > java.io.IOException: dataSize of 1249463589142530 starting at 5604968 would > be larger than file > /data/3/cassandra/data/users/global_user/users-global_user-ib-1550-Data.db > length 14017479 > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:168) > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:83) > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:69) > at > org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:177) > at > org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:152) > at > org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:139) > at > org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:36) > at > org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer$1.runMayThrow(ParallelCompactionIterable.java:288) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at java.lang.Thread.run(Thread.java:722) > Caused by: java.io.IOException: dataSize of 1249463589142530 starting at > 5604968 would be larger than file > /data/3/cassandra/data/users/global_user/users-global_user-ib-1550-Data.db > length 14017479 > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:123) > ... 9 more > > --001a1133d7b833ba7204e3345aa9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
you can try nodetool scrub. if it does not work, try repai= r then cleanup. had this issue a few weeks back but our version is 1.0.x


On Mon, Au= g 5, 2013 at 8:12 AM, Keith Wright <kwright@nanigans.com>= wrote:
Re-sending hoping to get some he= lp. =A0Any ideas would be much appreciated!

From: Keith Wright <kwright@nanigans.com>=
Date: Friday, August 2, 2013 3= :01 PM
To: "user@cassandra.apache.org" = <user@cas= sandra.apache.org>
Subject: org.apache.cassandra.io.s= stable.CorruptSSTableException

Hi all,

=A0 =A0We just added a node to our cluster= (1.2.4 Vnodes) and they appear to be running well exception I see that the= new node is not making any progress compacting one of the CF. =A0The excep= tion below is generated. =A0My assumption is that the only way to handle this is to stop the node, delete the file in question, restart, and run re= pair.

Thoughts?

org.apach= e.cassandra.io.sstable.CorruptSSTableException: java.io.IOException: dataSi= ze of 1249463589142530 starting at 5604968 would be larger than file /data/= 3/cassandra/data/users/global_user/users-global_user-ib-1550-Data.db length= 14017479
=A0 =A0 =A0 =A0 at org.apache.cassandra.io.sstable.SSTableIdentityIter= ator.<init>(SSTableIdentityIterator.java:168)
=A0 =A0 =A0 = =A0 at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>= (SSTableIdentityIterator.java:83)
=A0 =A0 =A0 =A0 at org.apache.cassandra.io.sstable.SSTableIdentityIter= ator.<init>(SSTableIdentityIterator.java:69)
=A0 =A0 =A0 = =A0 at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.n= ext(SSTableScanner.java:177)
=A0 =A0 =A0 =A0 at org.apache.cassandra.io.sstable.SSTableScanner$KeyS= canningIterator.next(SSTableScanner.java:152)
=A0 =A0 =A0 =A0 at = org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:139= )
=A0 =A0 =A0 =A0 at org.apache.cassandra.io.sstable.SSTableScanner.next(SSTa= bleScanner.java:36)
=A0 =A0 =A0 =A0 at org.apache.cassandra.db.co= mpaction.ParallelCompactionIterable$Deserializer$1.runMayThrow(ParallelComp= actionIterable.java:288)
=A0 =A0 =A0 =A0 at org.apache.cassandra.utils.WrappedRunnable.run(Wrap= pedRunnable.java:28)
=A0 =A0 =A0 =A0 at java.lang.Thread.run(Thre= ad.java:722)
Caused by: java.io.IOException: dataSize of 12494635= 89142530 starting at 5604968 would be larger than file /data/3/cassandra/da= ta/users/global_user/users-global_user-ib-1550-Data.db length 14017479
=A0 =A0 =A0 =A0 at org.apache.cassandra.io.sstable.SSTableIdentityIter= ator.<init>(SSTableIdentityIterator.java:123)
=A0 =A0 =A0 = =A0 ... 9 more


--001a1133d7b833ba7204e3345aa9--