cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jji...@apache.org
Subject [1/6] cassandra git commit: On log replay, the last segment header may be truncated (or null). We should tolerate failure to read the header the same way we tolerate truncation of any of the rest of the final segment
Date Tue, 05 Sep 2017 18:49:46 GMT
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 ef5ac1a4a -> 0493545dd
  refs/heads/cassandra-3.11 d44a0d25b -> 3f5f70281
  refs/heads/trunk 733ea88fa -> fa36044a5


On log replay, the last segment header may be truncated (or null). We should tolerate failure
to read the header the same way we tolerate truncation of any of the rest of the final segment

Patch by Jeff Jirsa; Reviewed by Branimir Lambov for CASSANDRA-11995


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0493545d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0493545d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0493545d

Branch: refs/heads/cassandra-3.0
Commit: 0493545dd08d29c34d757bb2f1d90052d03d24c6
Parents: ef5ac1a
Author: Jeff Jirsa <jeff@jeffjirsa.net>
Authored: Fri Mar 17 13:54:03 2017 -0700
Committer: Jeff Jirsa <jjirsa@apple.com>
Committed: Tue Sep 5 11:46:03 2017 -0700

----------------------------------------------------------------------
 CHANGES.txt                                            |  1 +
 .../cassandra/db/commitlog/CommitLogReplayer.java      |  4 +++-
 .../apache/cassandra/db/commitlog/CommitLogTest.java   | 13 ++++++++++++-
 3 files changed, 16 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0493545d/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index 853bf61..a3eccf2 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Better handle corrupt final commitlog segment (CASSANDRA-11995)
  * StreamingHistogram is not thread safe (CASSANDRA-13756)
  * Fix MV timestamp issues (CASSANDRA-11500)
  * Better tolerate improperly formatted bcrypt hashes (CASSANDRA-13626) 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0493545d/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index 205c36a..b3b26dd 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -355,7 +355,9 @@ public class CommitLogReplayer
                 desc = null;
             }
             if (desc == null) {
-                handleReplayError(false, "Could not read commit log descriptor in file %s",
file);
+                // Presumably a failed CRC or other IO error occurred, which may be ok if
it's the last segment
+                // where we tolerate (and expect) truncation
+                handleReplayError(tolerateTruncation, "Could not read commit log descriptor
in file %s", file);
                 return;
             }
             if (segmentId != desc.id)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0493545d/test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java
----------------------------------------------------------------------
diff --git a/test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java b/test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java
index 90dc258..1543415 100644
--- a/test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java
+++ b/test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java
@@ -117,13 +117,24 @@ public class CommitLogTest
     @Test
     public void testRecoveryWithEmptyLog() throws Exception
     {
+        // The first empty file we expect to throw as it's invalid
+        // We need to pass the second as well, because allowTruncation will be set to true
for the final segment
         runExpecting(() -> {
-            CommitLog.instance.recover(new File[]{ tmpFile(CommitLogDescriptor.current_version)
});
+            CommitLog.instance.recover(new File[]{
+                    tmpFile(CommitLogDescriptor.current_version),
+                    tmpFile(CommitLogDescriptor.current_version)  });
             return null;
         }, CommitLogReplayException.class);
     }
 
     @Test
+    public void testRecoveryWithEmptyFinalLog() throws Exception
+    {
+        // Even though it's empty, it's the last commitlog segment, so allowTruncation=true
should allow it to pass
+        CommitLog.instance.recover(new File[]{ tmpFile(CommitLogDescriptor.current_version)
 });
+    }
+
+    @Test
     public void testRecoveryWithEmptyLog20() throws Exception
     {
         CommitLog.instance.recover(new File[]{ tmpFile(CommitLogDescriptor.VERSION_20) });


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message