cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Loegering (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10315) Cassandra nodes shutting down on COMMIT-LOG-ALLOCATOR error
Date Mon, 14 Sep 2015 18:16:46 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743957#comment-14743957
] 

David Loegering commented on CASSANDRA-10315:
---------------------------------------------

How was this issue fixed and in what version?  What is causing the problem?  I looked at the
duplicate and the errors do not match the errors we are seeing.

> Cassandra nodes shutting down on COMMIT-LOG-ALLOCATOR error
> -----------------------------------------------------------
>
>                 Key: CASSANDRA-10315
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10315
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: 16 GB, Cassandra 2.0.15, 
>            Reporter: David Loegering
>
> After migrating from 2.0.9 to 2.0.15  all nodes on multiple clusters Cassandra nodes
shutting themselves down every 24-48 hours.  The error reported is:
> {code}
> ERROR [COMMIT-LOG-ALLOCATOR] 2015-09-11 17:14:48,034 CommitLog.java (line 420) Failed
to allocate new commit log segments. Commit disk failure policy is stop; terminating thread
> java.lang.AssertionError: attempted to delete non-existing file CommitLog-3-1441961724221.log
>             at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:113)
>             at org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:161)
>             at org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:228)
>             at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:99)
>             at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>             at java.lang.Thread.run(Unknown Source)
> {code}
> We’ve seen this error before, but now with DEBUG / TRACE on, we see that this file
has 18 entries starting at 5:14pm that look something like this:
> {code}
> DEBUG [COMMIT-LOG-WRITER] 2015-09-11 17:14:45,372 CommitLog.java (line 245) Not safe
to delete commit log segment CommitLogSegment(/opt/osi/monarch/chronus/resources/cassandra/commitlog/CommitLog-3-1441961724221.log);
dirty is soe (a77b7765-1e3b-30eb-9f46-2cac8dfe1ac7), min_max_avg_hourly (f86973a2-f5e6-36b6-9d7f-7fc4e109fb6e),
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message