Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7C18A105EC for ; Wed, 24 Jul 2013 20:55:01 +0000 (UTC) Received: (qmail 17371 invoked by uid 500); 24 Jul 2013 20:54:59 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 17325 invoked by uid 500); 24 Jul 2013 20:54:59 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 17317 invoked by uid 99); 24 Jul 2013 20:54:59 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jul 2013 20:54:59 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of paulingalls@gmail.com designates 209.85.160.49 as permitted sender) Received: from [209.85.160.49] (HELO mail-pb0-f49.google.com) (209.85.160.49) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jul 2013 20:54:51 +0000 Received: by mail-pb0-f49.google.com with SMTP id jt11so10115359pbb.8 for ; Wed, 24 Jul 2013 13:54:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:message-id:mime-version:subject:date:references :to:in-reply-to:x-mailer; bh=Q5pMeqavVU9LrBgAFj/fR0bWEXT/5fwpyXgsM0By7O0=; b=j1PCrigEzXMPxe/gi7xu+l11oW/fWzy2yNnIUjX1mFMuxbC5/SxbHUBIycQ4uYrjep 9hQHfaY7YvI/45byHQ8h/fmciAwvjUHox9545RdWGbLYd0GbX/6xGGoRgUBeKmmWHLSW 8xmYyzjK4QsS0qxlfD1QRIlSvexeaWv+kdVuUKaAfDvQac3SDQnvec9CDiwfvCB7zB7D a44ypUj86jn7w5IMptulBScqrsIop/Ojhz5XxhjfsNI97j4+VkPX1DuTmY9gGwqEV6W8 1hWEDKhEzF5iioEWj/wk3ZWexHRMzeLDj/vGQNXQWhwNTVaFC0uqIOj5C7gF4nO3JwPQ FtCw== X-Received: by 10.68.241.198 with SMTP id wk6mr44034741pbc.165.1374699270079; Wed, 24 Jul 2013 13:54:30 -0700 (PDT) Received: from [192.168.15.76] (184-78-184-52.war.clearwire-wmx.net. [184.78.184.52]) by mx.google.com with ESMTPSA id om2sm49710729pbb.34.2013.07.24.13.54.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 24 Jul 2013 13:54:29 -0700 (PDT) From: Paul Ingalls Content-Type: multipart/alternative; boundary="Apple-Mail=_423A3BFD-A41D-4878-847B-E39EE068E44B" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: unable to compact large rows Date: Wed, 24 Jul 2013 13:54:26 -0700 References: To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1508) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_423A3BFD-A41D-4878-847B-E39EE068E44B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 It is pretty much every row that hits the large threshold. I don't = think I can delete every row that hits that=85 you can see the db size in the stack trace, do you want a different type = of size? On Jul 24, 2013, at 11:07 AM, Jason Wee wrote: > Would it possible to delete this row and reinsert this row? By the = way, how large is that one row? >=20 > Jason >=20 >=20 > On Wed, Jul 24, 2013 at 9:23 AM, Paul Ingalls = wrote: > I'm getting constant exceptions during compaction of large rows. In = fact, I have not seen one work, even starting from an empty DB. As soon = as I start pushing in data, when a row hits the large threshold, it = fails compaction with this type of stack trace: >=20 > INFO [CompactionExecutor:6] 2013-07-24 01:17:53,592 = CompactionController.java (line 156) Compacting large row = fanzo/tweets_by_id:352567939972603904 (153360688 bytes) incrementally > ERROR [CompactionExecutor:6] 2013-07-24 01:18:12,496 = CassandraDaemon.java (line 192) Exception in thread = Thread[CompactionExecutor:6,1,main] > java.lang.AssertionError: incorrect row data size 5722610 written to = /mnt/datadrive/lib/cassandra/data/fanzo/tweets_by_id/fanzo-tweets_by_id-tm= p-ic-1453-Data.db; correct is 5767384 > at = org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:16= 2) > at = org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.j= ava:162) > at = org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnab= le.java:48) > at = org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at = org.apache.cassandra.db.compaction.CompactionTask.executeInternal(Compacti= onTask.java:58) > at = org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(Abstract= CompactionTask.java:60) > at = org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionT= ask.run(CompactionManager.java:211) > at = java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at = java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at = java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:= 1145) > at = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java= :615) > at java.lang.Thread.run(Thread.java:724) >=20 > I'm not sure what to do or where to look. Help=85:) >=20 > Thanks, >=20 > Paul >=20 >=20 >=20 --Apple-Mail=_423A3BFD-A41D-4878-847B-E39EE068E44B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 peichieh@gmail.com> = wrote:
Would it possible to delete this row and = reinsert this row? By the way, how large is that one = row?

Jason


On Wed, Jul 24, = 2013 at 9:23 AM, Paul Ingalls <paulingalls@gmail.com> wrote:
I'm getting constant = exceptions during compaction of large rows.  In fact, I have not = seen one work, even starting from an empty DB.  As soon as I start = pushing in data, when a row hits the large threshold, it fails = compaction with this type of stack trace:

 INFO [CompactionExecutor:6] 2013-07-24 01:17:53,592 = CompactionController.java (line 156) Compacting large row = fanzo/tweets_by_id:352567939972603904 (153360688 bytes) = incrementally
ERROR [CompactionExecutor:6] 2013-07-24 01:18:12,496 = CassandraDaemon.java (line 192) Exception in thread = Thread[CompactionExecutor:6,1,main]
java.lang.AssertionError: incorrect row data size 5722610 written to = /mnt/datadrive/lib/cassandra/data/fanzo/tweets_by_id/fanzo-tweets_by_id-tm= p-ic-1453-Data.db; correct is 5767384
        at = org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:16= 2)
        at = org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.j= ava:162)
        at = org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnab= le.java:48)
        at = org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)         at = org.apache.cassandra.db.compaction.CompactionTask.executeInternal(Compacti= onTask.java:58)
        at = org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(Abstract= CompactionTask.java:60)
        at = org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionT= ask.run(CompactionManager.java:211)
        at = java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at = java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at = java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at = java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:= 1145)
        at = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java= :615)
        at java.lang.Thread.run(Thread.java:724)

I'm not sure what to do or where to look.  Help=85:)

Thanks,

Paul




= --Apple-Mail=_423A3BFD-A41D-4878-847B-E39EE068E44B--