cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9478) SSTables are not always properly marked suspected
Date Mon, 01 Jun 2015 16:12:18 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567499#comment-14567499
] 

Benedict commented on CASSANDRA-9478:
-------------------------------------

One nit: should just use {{replacement.isSuspect.set()}}, not {{replacement.isSuspect.getAndSet()}}
 (this latter involves an atomic operation unnecessarily). In fact we could happily use {{lazySet}},
but that's not so important.

We could also switch the early open interval and ColumnIndex size very low, so that during
BlacklistingCompactionTest it's more likely to have happened at some point in 2.2 (and so
we would at least hopefully get an intermittent failure).

> SSTables are not always properly marked suspected
> -------------------------------------------------
>
>                 Key: CASSANDRA-9478
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9478
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Sylvain Lebresne
>            Assignee: Sylvain Lebresne
>             Fix For: 2.0.x
>
>
> There is 2 problems:
> # During compaction, {{SSTableIdentityIterator}} doesn't always mark sstables suspect
(nor always throws {{CorruptSSTableException}} in 2.0) This affects versions at least since
2.0 and up to trunk.
> # Since 2.1, with CASSANDRA-916, {{SSTableReader.cloneWithNewStart}} doesn't preserve
the isSuspect flag. This a problem because when a {{CorruptSSTableException}} is thrown, the
{{SSTableRewriter}} is aborted, which replace {{SSTableReader}} by a clone of themselves and
for the sstables that have just been marked suspected this (wrongly) clears the flag.
> The reason none of this has be caught so far is that the test that tests sstable blacklisting,
{{BlacklistingCompactionsTest}}, always corrupts the very beginning of sstables (the first
3 bytes). This conspire in hiding the 2 problems above because:
> # the {{CorruptSSTableException}} is always thrown when we read the very first partition
key, which is actually properly covered in {{SSTableIdentityIterator}}.
> # due to {{MergeIterator}} greediness, the exception is throw in {{CompactionTask.runMayThrow}}
on the creation of the iterator (on {{Iterator<AbstractCompactedRow> iter = ci.iterator()}}),
which is before the {{SSTableRewriter}} is even created, and so it's not aborted (and the
suspected flag is not cleared).
> I've made some simple changes to {{BlacklistingCompactionsTest}} so it corrupts files
a little bit more randomly. At least on 2.1, the updated test pretty reliably find the 2 problems
above. On 2.2/trunk however, this doesn't find the 2nd one because since CASSANDRA-8568, {{SSTableRewriter.doAbort}}
do nothing if no early opening was actually perform, which will be the case in the test. We
would need a fancier test that ensure the corruption is far enough in the file that some early
opening has been done when it's thrown. But I don't have time for writing that test right
now so I'm gonna push that to a follow-up ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message