cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Balaji Venkatesan <venkatesan.bal...@gmail.com>
Subject Re: nodetool repair failure
Date Thu, 29 Jun 2017 19:05:23 GMT
Thanks. I tried with trace option and there is not much info. Here are the
few log lines just before it failed.


[2017-06-29 19:01:54,969] /xx.xx.xx.93: Sending REPAIR_MESSAGE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Adding to repair_history memtable
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Enqueuing response to /xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:01:54,969] /xx.xx.xx.92: Sending REQUEST_RESPONSE message to
/xx.xx.xx.91
[2017-06-29 19:02:04,842] Some repair failed
[2017-06-29 19:02:04,848] Repair command #1 finished in 1 minute 2 seconds
error: Repair job has failed with the error message: [2017-06-29
19:02:04,842] Some repair failed
-- StackTrace --
java.lang.RuntimeException: Repair job has failed with the error message:
[2017-06-29 19:02:04,842] Some repair failed
at org.apache.cassandra.tools.RepairRunner.progress(RepairRunner.java:116)
at org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListene
r.handleNotification(JMXNotificationProgressListener.java:77)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.
dispatchNotification(ClientNotifForwarder.java:583)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(
ClientNotifForwarder.java:533)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(
ClientNotifForwarder.java:452)
at com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(
ClientNotifForwarder.java:108)



FYI I am running repair from xx.xx.xx.91 node and its a 5 node cluster
xx.xx.xx.91-xx.xx.xx.95

On Wed, Jun 28, 2017 at 5:16 PM, Akhil Mehra <akhilmehra@gmail.com> wrote:

> nodetool repair has a trace option
>
> nodetool repair -tr yourkeyspacename
>
> see if that provides you with additional information.
>
> Regards,
> Akhil
>
> On 28/06/2017, at 2:25 AM, Balaji Venkatesan <venkatesan.balaji@gmail.com>
> wrote:
>
>
> We use Apache Cassandra 3.10-13
>
> On Jun 26, 2017 8:41 PM, "Michael Shuler" <michael@pbandjelly.org> wrote:
>
> What version of Cassandra?
>
> --
> Michael
>
> On 06/26/2017 09:53 PM, Balaji Venkatesan wrote:
> > Hi All,
> >
> > When I run nodetool repair on a keyspace I constantly get  "Some repair
> > failed" error, there are no sufficient info to debug more. Any help?
> >
> > Here is the stacktrace
> >
> > ======================================================================
> > [2017-06-27 02:44:34,275] Some repair failed
> > [2017-06-27 02:44:34,279] Repair command #3 finished in 33 seconds
> > error: Repair job has failed with the error message: [2017-06-27
> > 02:44:34,275] Some repair failed
> > -- StackTrace --
> > java.lang.RuntimeException: Repair job has failed with the error
> > message: [2017-06-27 02:44:34,275] Some repair failed
> > at org.apache.cassandra.tools.RepairRunner.progress(RepairRunne
> r.java:116)
> > at
> > org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.
> handleNotification(JMXNotificationProgressListener.java:77)
> > at
> > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetche
> r.dispatchNotification(ClientNotifForwarder.java:583)
> > at
> > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetche
> r.doRun(ClientNotifForwarder.java:533)
> > at
> > com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetche
> r.run(ClientNotifForwarder.java:452)
> > at
> > com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecu
> tor$1.run(ClientNotifForwarder.java:108)
> > ======================================================================
> >
> >
> > --
> > Thanks,
> > Balaji Venkatesan.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
>
>
>


-- 
Thanks,
Balaji Venkatesan.

Mime
View raw message