cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Muga Nishizawa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2752) repair fails with java.io.EOFException
Date Fri, 10 Jun 2011 03:50:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046984#comment-13046984
] 

Muga Nishizawa commented on CASSANDRA-2752:
-------------------------------------------

It seems tmp files (e.g. XXX-tmp-XXX-Data.db) that receiver node creates during repair process
are broken.  EOFException occurs while RowIndexer is reading the broken files.  

According to result of scrub command, data files on sender nodes are not broken.  

> repair fails with java.io.EOFException
> --------------------------------------
>
>                 Key: CASSANDRA-2752
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2752
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.8.0
>            Reporter: Terje Marthinussen
>
> Issuing repair on node 1  (1.10.42.81) in a cluster quickly fails with
> INFO [AntiEntropyStage:1] 2011-06-09 19:02:47,999 AntiEntropyService.java (line 234)
Queueing comparison #<Differencer #<TreeRequest manual-repair-0c17c5f9-583f-4a31-a6d4-a9e7306fb46e,
/1
> .10.42.82, (JP,XXX), (Token(bytes[6e]),Token(bytes[313039])]>>
>  INFO [AntiEntropyStage:1] 2011-06-09 19:02:48,026 AntiEntropyService.java (line 468)
Endpoints somewhere/1.10.42.81 and /1.10.42.82 have 2 range(s) out of sync for (JP,XXX) on
(Token(bytes[6e]),Token(bytes[313039])]
>  INFO [AntiEntropyStage:1] 2011-06-09 19:02:48,026 AntiEntropyService.java (line 485)
Performing streaming repair of 2 ranges for #<TreeRequest manual-repair-0c17c5f9-583f-4a31-a6d4-a9e7306
> fb46e, /1.10.42.82, (JP,XXX), (Token(bytes[6e]),Token(bytes[313039])]>
>  INFO [AntiEntropyStage:1] 2011-06-09 19:02:48,030 StreamOut.java (line 173) Stream context
metadata [/data/cassandra/node0/data/JP/XXX-g-3-Data.db sections=1 progress=0/36592 - 0%],
1 sstables.
>  INFO [AntiEntropyStage:1] 2011-06-09 19:02:48,031 StreamOutSession.java (line 174) Streaming
to /1.10.42.82
> ERROR [CompactionExecutor:9] 2011-06-09 19:02:48,970 AbstractCassandraDaemon.java (line
113) Fatal exception in thread Thread[CompactionExecutor:9,1,main]
> java.io.EOFException
>         at java.io.RandomAccessFile.readInt(RandomAccessFile.java:725)
>         at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.doIndexing(SSTableWriter.java:457)
>         at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:364)
>         at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
>         at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:1099)
>         at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:1090)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> On .82
> ERROR [CompactionExecutor:12] 2011-06-09 19:02:48,051 AbstractCassandraDaemon.java (line
113) Fatal exception in thread Thread[CompactionExecutor:12,1,main]
> java.io.EOFException
>         at java.io.RandomAccessFile.readInt(RandomAccessFile.java:725)
>         at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.doIndexing(SSTableWriter.java:457)
>         at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:364)
>         at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
>         at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:1099)
>         at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:1090)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> ERROR [Thread-132] 2011-06-09 19:02:48,051 AbstractCassandraDaemon.java (line 113) Fatal
exception in thread Thread[Thread-132,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.EOFException
>         at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:152)
>         at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
>         at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:155)
>         at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:93)
> Caused by: java.util.concurrent.ExecutionException: java.io.EOFException
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:136)
>         ... 3 more
> Caused by: java.io.EOFException
>         at java.io.RandomAccessFile.readInt(RandomAccessFile.java:725)
>         at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.doIndexing(SSTableWriter.java:457)
>         at org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:364)
>         at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
>         at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:1099)
>         at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:1090)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Looks to me like the receiving side fails first.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message