cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Terry Cumaranatunge (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-5535) Manifest file not fsynced
Date Fri, 03 May 2013 15:54:17 GMT
Terry Cumaranatunge created CASSANDRA-5535:
----------------------------------------------

             Summary: Manifest file not fsynced
                 Key: CASSANDRA-5535
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5535
             Project: Cassandra
          Issue Type: Bug
    Affects Versions: 1.2.3
         Environment: RHEL 6.4
java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

            Reporter: Terry Cumaranatunge
            Priority: Critical


We had several cases where the the manifest file would get corrupted when doing power reset
tests or iLO resets to mimic power failure scenarios, ungraceful resets, kernel panics etc.
It wasn't clear at the time where the problem was, but I think the data below shows that Cassandra
is missing an fsync call to the manifest file prior to closing it. This particular stack trace
from below is on Cassandra 1.2.4.
The trace was captured using strace options:

strace -f -p 2200 -e trace=open,close,write,fsync,fdatasync,rename
[pid 9710] open("/opt/mp/storage/persistent/cassandra/cassandra-lib/data/MSA/subinfo/subinfo-tmp.json",
O_WRONLY|O_CREAT|O_TRUNC, 0666) = 238
[pid 9710] write(238, "{\n \"generations\" : [ {\n \"gen"..., 3996) = 3996
[pid 9710] write(238, "14, 263161, 263484, 270816, 2593"..., 3996) = 3996
[pid 9710] write(238, "275136, 275137, 275138, 275139, "..., 1173) = 1173
[pid 9710] close(238) = 0
[pid 9710] rename("/opt/mp/storage/persistent/cassandra/cassandra-lib/data/MSA/subinfo/subinfo.json"
, "/opt/mp/storage/persistent/cassandra/cassandra-lib/data/MSA/subinfo/subinfo-old.json")
= 0
[pid 9710] rename("/opt/mp/storage/persistent/cassandra/cassandra-lib/data/MSA/subinfo/subinfo-tmp.j
son", "/opt/mp/storage/persistent/cassandra/cassandra-lib/data/MSA/subinfo/subinfo.json")
= 0


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message