cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Lourie (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-14080) Handling 0 size hint files during start
Date Tue, 05 Dec 2017 00:27:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16277796#comment-16277796
] 

Alex Lourie edited comment on CASSANDRA-14080 at 12/5/17 12:26 AM:
-------------------------------------------------------------------

[~iamaleksey] That should prevent empty hint files to be written as much as C* is concerned.
The problem is that we have no way to reproduce creation of the empty hint files and for all
we know these hint files could have been created from external circumstances such as corrupt
disks. We should still handle 0 length files but this isn't a ticket for a long drawn out
investigation with possibly no results.


was (Author: alourie):
[~iamaleksey] while that should prevent empty hint files to be written, there could be other,
external, circumstances for their existence. The patch in this ticket is to make sure C* doesn't
crash if that happens.

> Handling 0 size hint files during start
> ---------------------------------------
>
>                 Key: CASSANDRA-14080
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14080
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Hints
>            Reporter: Aleksandr Ivanov
>            Assignee: Alex Lourie
>
> Continuation of CASSANDRA-12728 bug.
> Problem: Cassandra didn't start due to 0 size hints files
> Log form v3.0.14:
> {code:java}
> INFO  [main] 2017-11-28 19:10:13,554 StorageService.java:575 - Cassandra version: 3.0.14
> INFO  [main] 2017-11-28 19:10:13,555 StorageService.java:576 - Thrift API version: 20.1.0
> INFO  [main] 2017-11-28 19:10:13,555 StorageService.java:577 - CQL supported versions:
3.4.0 (default: 3.4.0)
> ERROR [main] 2017-11-28 19:10:13,592 CassandraDaemon.java:710 - Exception encountered
during startup
> org.apache.cassandra.io.FSReadError: java.io.EOFException
>         at org.apache.cassandra.hints.HintsDescriptor.readFromFile(HintsDescriptor.java:142)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
~[na:1.8.0_141]
>         at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[na:1.8.0_141]
>         at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_141]
>         at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
~[na:1.8.0_141]
>         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[na:1.8.0_141]
>         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
~[na:1.8.0_141]
>         at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
~[na:1.8.0_141]
>         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:1.8.0_141]
>         at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[na:1.8.0_141]
>         at org.apache.cassandra.hints.HintsCatalog.load(HintsCatalog.java:65) ~[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.hints.HintsService.<init>(HintsService.java:88)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.hints.HintsService.<clinit>(HintsService.java:63)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.service.StorageProxy.<clinit>(StorageProxy.java:121)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at java.lang.Class.forName0(Native Method) ~[na:1.8.0_141]
>         at java.lang.Class.forName(Class.java:264) ~[na:1.8.0_141]
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:585)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:570)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:346)
[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
[apache-cassandra-3.0.14.jar:3.0.14]
> Caused by: java.io.EOFException: null
>         at java.io.RandomAccessFile.readInt(RandomAccessFile.java:803) ~[na:1.8.0_141]
>         at org.apache.cassandra.hints.HintsDescriptor.deserialize(HintsDescriptor.java:237)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         at org.apache.cassandra.hints.HintsDescriptor.readFromFile(HintsDescriptor.java:138)
~[apache-cassandra-3.0.14.jar:3.0.14]
>         ... 20 common frames omitted
> {code}
> After several 0 size hints files deletion Cassandra started successfully.
> Jeff Jirsa added a comment - Yesterday
> Aleksandr Ivanov can you open a new JIRA and link it back to this one? It's possible
that the original patch didn't consider 0 byte files (I don't have time to go back and look
at the commit, and it was long enough ago that I've forgotten) - were all of your files 0
bytes?
> Not all, 8..10 hints files were with 0 size.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message