cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "david (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9668) RepairException when trying to run concurrent repair -pr
Date Sun, 28 Jun 2015 15:16:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14604718#comment-14604718
] 

david commented on CASSANDRA-9668:
----------------------------------

Yes, here is corrosponding error:

{noformat}
ERROR [ValidationExecutor:19] 2015-06-28 09:33:12,029 CompactionManager.java:972 - Cannot
start multiple repair sessions over the same sstables
ERROR [ValidationExecutor:19] 2015-06-28 09:33:12,029 Validator.java:245 - Failed creating
a merkle tree for [repair #b1c30fe0-1d78-11e5-aec7-4f05493cbe02 on keyspace/data, (9062648853864216757,9072201154757474095]],
/172.31.46.189 (see log for details)
ERROR [ValidationExecutor:19] 2015-06-28 09:33:12,029 CassandraDaemon.java:223 - Exception
in thread Thread[ValidationExecutor:19,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same sstables
        at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:973)
~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:94)
~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:623)
~[apache-cassandra-2.1.7.jar:2.1.7]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_40]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_40]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_40]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
{noformat}
This suggest running concurrent repairs (with -pr) is not possible. Is this true?

> RepairException when trying to run concurrent repair -pr
> --------------------------------------------------------
>
>                 Key: CASSANDRA-9668
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9668
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Cassandra 2.1.7
>            Reporter: david
>            Assignee: Yuki Morishita
>            Priority: Critical
>             Fix For: 2.1.x
>
>
> Was on 2.1.3 having very similar issues to those described in:
> https://issues.apache.org/jira/browse/CASSANDRA-9266
> I updated to 2.1.7, more for some other fixes, but now if I try and run concurrent repairs
(different boxes) consistently get:
> {noformat}
> ERROR [Thread-14156] 2015-06-28 09:33:12,616 StorageService.java:2959 - Repair session
b1e67660-1d78-11e5-aec7-4f05493cbe02 for range (-4660677346721084182,-4658765298409301171]
failed with error org.apache.cassandra.exceptions.RepairException: [repair #b1e67660-1d78-11e5-aec7-4f05493cbe02
on keyspace/data, (-4660677346721084182,-4658765298409301171]] Validation failed in /172.31.13.127
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException:
[repair #b1e67660-1d78-11e5-aec7-4f05493cbe02 on keyspace/data, (-4660677346721084182,-4658765298409301171]]
Validation failed in /172.31.13.127
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122) [na:1.8.0_40]
>         at java.util.concurrent.FutureTask.get(FutureTask.java:192) [na:1.8.0_40]
>         at org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:2950)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) [apache-cassandra-2.1.7.jar:2.1.7]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_40]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_40]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
> Caused by: java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException:
[repair #b1e67660-1d78-11e5-aec7-4f05493cbe02 on keyspace/data, (-4660677346721084182,-4658765298409301171]]
Validation failed in /172.31.13.127
>         at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.jar:na]
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) [apache-cassandra-2.1.7.jar:2.1.7]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_40]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_40]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_40]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_40]
>         ... 1 common frames omitted
> Caused by: org.apache.cassandra.exceptions.RepairException: [repair #b1e67660-1d78-11e5-aec7-4f05493cbe02
on keyspace/data, (-4660677346721084182,-4658765298409301171]] Validation failed in /172.31.13.127
>         at org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:406)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:134)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         ... 3 common frames omitted
> {noformat}
> The specific repair command being issued:
> {noformat}
> nodetool repair -local -pr -inc -par -- keyspace &
> {noformat}
> It's a 15 box environment with a replication factor of 3.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message