cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Shuler (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (CASSANDRA-8333) Streaming Error during repair
Date Mon, 01 Dec 2014 17:59:12 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael Shuler reopened CASSANDRA-8333:
---------------------------------------

For the sake of testing mixed version repairs, I'm going to reopen.

> Streaming Error during repair
> -----------------------------
>
>                 Key: CASSANDRA-8333
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8333
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_55
>            Reporter: Andreas Schnitzerling
>         Attachments: system.log
>
>
> During repair, connections are closing and throwing exceptions. CPU is running on 100%,
when error occurs. My test-configuration is one node w/ 2.1.2 and 11 nodes w/ 2.0.11. If I
make repair either on 2.1 or 2.0 I get such an error. But if I have 2.0 everywhere istalled,
no error. 2.0 nodes make "endless" repair in that circumstance. Seems to be incompatibility
between 2.0 and 2.1. 
> {panel:title=system.log}
> ERROR [STREAM-OUT-/10.6.8.212] 2014-11-18 12:28:34,948 StreamSession.java:472 - [Stream
#0866dc80-6f16-11e4-bc5c-5fe413b6852c] Streaming error occurred
> java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert
> durch den Hostcomputer abgebrochen
> 	at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
> 	at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_55]
> 	at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) ~[na:1.7.0_55]
> 	at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_55]
> 	at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.7.0_55]
> 	at org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:346)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:326)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> ERROR [AntiEntropySessions:1] 2014-11-18 12:28:34,948 RepairSession.java:303 - [repair
#e10d0240-6f15-11e4-bc5c-5fe413b6852c] session completed with the following error
> org.apache.cassandra.exceptions.RepairException: [repair #e10d0240-6f15-11e4-bc5c-5fe413b6852c
on logdata/onlinedata, (-143721749331492309,-139544903266258032]] Sync failed between /10.9.9.241
and /10.6.8.212
> 	at org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:223) ~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:389)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:126)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) ~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55]
> 	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> {panel}
> Since in windows only parallel repair is possible, is there a way to throttle CPU-consumption?
I reduced rpc_X_threads to 4 and concurrent_reads/writes to 4. But no change. On other nodes
is C* 2.0.10 and nothing in their system.log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message