cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-9525) Commitlog allocation failure doesn't stop the entire node.
Date Mon, 13 Jul 2015 13:02:04 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benedict resolved CASSANDRA-9525.
---------------------------------
    Resolution: Fixed

[~adkhare]: the problem reported here is a duplicate of CASSANDRA-8515, which is where a fix
will be delivered. It only affects nodes during startup, though, so it isn't something that
should be affecting clusters with any regularity. If your issue doesn't match those characteristics,
please file a new JIRA with all of the information you have.

> Commitlog allocation failure doesn't stop the entire node.
> ----------------------------------------------------------
>
>                 Key: CASSANDRA-9525
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9525
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Jeremiah Jordan
>             Fix For: 2.1.x
>
>
> If there is an error opening a commit log segment, the thread dies, but it doesn't stop
the whole node.
> Got the following on a node:
> {noformat}
> ERROR [COMMIT-LOG-ALLOCATOR] 2015-05-29 17:06:45,722  CommitLog.java:397 - Failed managing
commit log segments. Commit disk failure policy is stop; terminating thread
> org.apache.cassandra.io.FSWriteError: java.io.FileNotFoundException: /var/lib/cassandra/commitlog/CommitLog-4-1432937194590.log
(Permission denied)
> 	at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:177)
~[cassandra-all-2.1.5.jar:2.1.5]
> 	at org.apache.cassandra.db.commitlog.CommitLogSegmentManager$4.call(CommitLogSegmentManager.java:397)
~[cassandra-all-2.1.5.jar:2.1.5]
> 	at org.apache.cassandra.db.commitlog.CommitLogSegmentManager$4.call(CommitLogSegmentManager.java:394)
~[cassandra-all-2.1.5.jar:2.1.5]
> 	at org.apache.cassandra.db.commitlog.CommitLogSegmentManager$1.runMayThrow(CommitLogSegmentManager.java:152)
~[cassandra-all-2.1.5.jar:2.1.5]
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) [cassandra-all-2.1.5.jar:2.1.5]
> 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/commitlog/CommitLog-4-1432937194590.log
(Permission denied)
> 	at java.io.RandomAccessFile.open0(Native Method) ~[na:1.8.0_45]
> 	at java.io.RandomAccessFile.open(RandomAccessFile.java:316) ~[na:1.8.0_45]
> 	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) ~[na:1.8.0_45]
> 	at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:155)
~[cassandra-all-2.1.5.jar:2.1.5]
> 	... 5 common frames omitted
> {noformat}
> And the node stayed kind of up, didn't notice something wrong until the node died OOM
because some threads were dead and others weren't.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message