Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AFDDF18428 for ; Wed, 14 Oct 2015 19:29:13 +0000 (UTC) Received: (qmail 45233 invoked by uid 500); 14 Oct 2015 19:29:05 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 45200 invoked by uid 500); 14 Oct 2015 19:29:05 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 45185 invoked by uid 99); 14 Oct 2015 19:29:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Oct 2015 19:29:05 +0000 Date: Wed, 14 Oct 2015 19:29:05 +0000 (UTC) From: "Alan Boudreault (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CASSANDRA-10519) RepairException: [repair #... on .../..., (...,...]] Validation failed in /w.x.y.z MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-10519?page=3Dcom.atl= assian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-10519: ---------------------------------------- Reproduced In: 2.2.2 [~gabor.auth] Are you able to reproduce this easily? If so, can you attach = a test case or the procedures. [~yukim] How critical would you qualify a repair failing that way? > RepairException: [repair #... on .../..., (...,...]] Validation failed in= /w.x.y.z > -------------------------------------------------------------------------= --------- > > Key: CASSANDRA-10519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1051= 9 > Project: Cassandra > Issue Type: Bug > Environment: CentOS 7, JDK 8u60, Cassandra 2.2.2 (upgraded from 2= .1.5) > Reporter: G=C3=A1bor Auth > > Sometimes the repair fails: > {code} > ERROR [Repair#3:1] 2015-10-14 06:22:56,490 CassandraDaemon.java:185 - Exc= eption in thread Thread[Repair#3:1,5,RMI Runtime] > com.google.common.util.concurrent.UncheckedExecutionException: org.apache= .cassandra.exceptions.RepairException: [repair #018adc70-723c-11e5-b0d8-6b2= 151e4d388 on keyspace/table, (2414492737393085601,27880539413409 > 54029]] Validation failed in /w.y.x.z > at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecke= d(Futures.java:1387) ~[guava-16.0.jar:na] > at com.google.common.util.concurrent.Futures.getUnchecked(Futures= .java:1373) ~[guava-16.0.jar:na] > at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:169) = ~[apache-cassandra-2.2.2.jar:2.2.2] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1142) ~[na:1.8.0_60] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:617) ~[na:1.8.0_60] > at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60] > Caused by: org.apache.cassandra.exceptions.RepairException: [repair #018a= dc70-723c-11e5-b0d8-6b2151e4d388 on keyspace/table, (2414492737393085601,27= 88053941340954029]] Validation failed in /w.y.x.z > at org.apache.cassandra.repair.ValidationTask.treeReceived(Valida= tionTask.java:64) ~[apache-cassandra-2.2.2.jar:2.2.2] > at org.apache.cassandra.repair.RepairSession.validationComplete(R= epairSession.java:183) ~[apache-cassandra-2.2.2.jar:2.2.2] > at org.apache.cassandra.service.ActiveRepairService.handleMessage= (ActiveRepairService.java:399) ~[apache-cassandra-2.2.2.jar:2.2.2] > at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(Re= pairMessageVerbHandler.java:163) ~[apache-cassandra-2.2.2.jar:2.2.2] > at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDelive= ryTask.java:66) ~[apache-cassandra-2.2.2.jar:2.2.2] > ... 3 common frames omitted > {code} > And here is the w.y.x.z side: > {code} > ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487 CompactionManager.ja= va:1053 - Cannot start multiple repair sessions over the same sstables > ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,487 Validator.java:246 -= Failed creating a merkle tree for [repair #018adc70-723c-11e5-b0d8-6b2151e= 4d388 on keyspace/table, (2414492737393085601,2788053941340954029]], /a.b.c= .d (see log for details) > ERROR [ValidationExecutor:7] 2015-10-14 06:22:56,488 CassandraDaemon.java= :185 - Exception in thread Thread[ValidationExecutor:7,1,main] > java.lang.RuntimeException: Cannot start multiple repair sessions over th= e same sstables > at org.apache.cassandra.db.compaction.CompactionManager.doValidat= ionCompaction(CompactionManager.java:1054) ~[apache-cassandra-2.2.2.jar:2.2= .2] > at org.apache.cassandra.db.compaction.CompactionManager.access$70= 0(CompactionManager.java:86) ~[apache-cassandra-2.2.2.jar:2.2.2] > at org.apache.cassandra.db.compaction.CompactionManager$10.call(C= ompactionManager.java:652) ~[apache-cassandra-2.2.2.jar:2.2.2] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:= 1.8.0_60] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1142) ~[na:1.8.0_60] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:617) [na:1.8.0_60] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] > ... > ERROR [Reference-Reaper:1] 2015-10-14 06:23:21,439 Ref.java:187 - LEAK DE= TECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@74fc05= 4a) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceT= idier@1949471967:/home/cassandra/dsc-cassandra-2.2.2/bin/../data/data/keysp= ace/table-b15521b0621111e4bbedcdee5e027297/la-1195-big was not released bef= ore the reference was garbage collected > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)