cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12261) dtest failure in write_failures_test.TestWriteFailures.test_thrift
Date Thu, 28 Jul 2016 04:36:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396899#comment-15396899
] 

Stefania commented on CASSANDRA-12261:
--------------------------------------

This test seems to have failed only once with this specific failure and it cannot be reproduced
locally. [~mambocab], are you aware of any other tests with the same problem or a reasonably
reliable way to reproduce this?

The sstable transaction tidier deletes the sstable data file but at present it does not check
if the data file exists. I've added this check and, if the sstable was not new, a new error
message that should tell us if the sstable tidier ran without data file. After all sstable
tidiers have run, the transaction tidier does another pass to ensure no content files are
left before deleting the actual transaction log file. Here we were missing a folder sync,
so I think it's not 100% impossible that {{record.getExistingFiles()}} returned a wrong result
including files that were deleted just before by the sstable tidiers, although this wouldn't
explain why only data files were reported as non existing. I've added the folder sync call.
Unless the same file is involved in two different transactions, which shouldn't happen, there
shouldn't be any race between the call to record.getExistingFiles() and LogTransaction::delete
in LogFile.deleteRecordFiles().

I've also added the exception call stack in LogTransaction.delete(), but only to the debug
log file.

I've prepared the patches fro 3.9 and trunk:

||3.9||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/12261-3.9]|[patch|https://github.com/stef1927/cassandra/commits/12261]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12261-3.9-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12261-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12261-3.9-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12261-dtest/]|

This problem could technically happen in 3.0 as well, but I would rather not disrupt 3.0 unless
we are sure there is an issue in the first place.

> dtest failure in write_failures_test.TestWriteFailures.test_thrift
> ------------------------------------------------------------------
>
>                 Key: CASSANDRA-12261
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12261
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Philip Thompson
>            Assignee: Stefania
>              Labels: dtest
>             Fix For: 3.x
>
>         Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, node2_debug.log,
node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_novnode_dtest/14/testReport/write_failures_test/TestWriteFailures/test_thrift
> Failure is
> {code}
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,127 LogTransaction.java:205 - Unable to
delete /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-2-big-Data.db
as it does not exist
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,334 LogTransaction.java:205 - Unable to
delete /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-15-big-Data.db
as it does not exist
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,337 LogTransaction.java:205 - Unable to
delete /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-31-big-Data.db
as it does not exist
> Unexpected error in node3 log, error: 
> ERROR [NonPeriodicTasks:1] 2016-07-20 07:09:52,339 LogTransaction.java:205 - Unable to
delete /tmp/dtest-CSPEFG/test/node3/data2/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09/mb-18-big-Data.db
as it does not exist
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message