cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Marsten (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-11995) Commitlog replaced with all NULs
Date Thu, 15 Dec 2016 15:31:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644485#comment-15644485
] 

John Marsten edited comment on CASSANDRA-11995 at 12/15/16 3:31 PM:
--------------------------------------------------------------------

We observed this phenomenon running Apache Cassandra 3.7 on Windows Server 2012 VM. The precise
history and state of the VM in question was unclear, but apparently the administrators took
a live volume snapshot, and after reverting to the snapshot, Cassandra would no longer start
due to the error above. In this case, the commit log file was 48MB of NUL bytes. The Cassandra
system log showed an ungraceful termination and a handful of warnings related to slow commit
log syncs, but no errors prior to the CommitLogReplayException on subsequent startups.

Cassandra was configured for batch mode commit log syncing with a window size of 50ms. It's
not clear whether the system was writing to Cassandra at the time of the snapshot.


was (Author: jmarsten):
We observed this phenomenon running Apache Cassandra 3.8 on Windows Server 2012 VM. The precise
history and state of the VM in question was unclear, but apparently the administrators took
a live snapshot, and after reverting to the snapshot, Cassandra would no longer start due
to the error above. In this case, the commit log file was 48MB of NUL bytes. The Cassandra
system log showed an ungraceful termination and a handful of warnings related to slow commit
log syncs, but no errors prior to the CommitLogReplayException on subsequent startups.

Cassandra was configured for batch mode commit log syncing with a window size of 50ms. It's
not clear whether the system was writing to Cassandra at the time of the snapshot.

> Commitlog replaced with all NULs
> --------------------------------
>
>                 Key: CASSANDRA-11995
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11995
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Windows 10 Enterprise 1511
> DataStax Cassandra Community Server 2.2.3
>            Reporter: James Howe
>
> I noticed this morning that Cassandra was failing to start, after being shut down on
Friday.
> {code}
> ERROR 09:13:37 Exiting due to error while processing commit log during initialization.
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: Could not
read commit log descriptor in file C:\Program Files\DataStax Community\data\commitlog\CommitLog-5-1465571056722.log
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:622)
[apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:302)
[apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147)
[apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:273) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:513) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622) [apache-cassandra-2.2.3.jar:2.2.3]
> {code}
> Checking the referenced file reveals it comprises 33,554,432 (32 * 1024 * 1024) NUL bytes.
> No logs (stdout, stderr, prunsrv) from the shutdown show any other issues and appear
exactly as normal.
> Is installed as a service via DataStax's distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message