cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9591) Scrub (recover) sstables even when -Index.db is missing
Date Mon, 06 Jul 2015 02:53:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614496#comment-14614496
] 

Stefania commented on CASSANDRA-9591:
-------------------------------------

[~Benedict], I've rebased and squashed all 3 patched and created a new one for trunk, given
that 2.2 did not automatically apply.

I agree on obsoleting the originals in the standalone scrubber, it's neater. 

On trunk we must retain the statistcs file as well, the data file alone is not sufficient.
I've changed the dtest to reflect this, pull request is [here|https://github.com/riptano/cassandra-dtest/pull/364].

On 2.2 I encountered a new NPE after rebasing, it was caused by the key cache being missing.
I changed the txn code to set up the key cache for the originals if they don't already have
one, which could be true when offline. You may want to double check this.

I also updated CHANGES.txt.

The CI results are here:
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-2.0-testall/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-2.0-dtest/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-2.1-testall/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-2.1-dtest/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-2.2-testall/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-2.2-dtest/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-testall/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9591-dtest/lastCompletedBuild/testReport/

There are several failures but the same is true for the unpatched branches. For a few tests
you need to go back several builds to find the same test failing on the unpatched branch or
you can just run it locally and it should pass. I've also launched the builds at least twice
to shake off some flakiness. 

If there are any failing tests that concern you do let me know.

> Scrub (recover) sstables even when -Index.db is missing
> -------------------------------------------------------
>
>                 Key: CASSANDRA-9591
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9591
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: mck
>            Assignee: mck
>              Labels: benedict-to-commit, sstablescrub
>             Fix For: 2.0.x
>
>         Attachments: 9591-2.0.txt, 9591-2.1.txt
>
>
> Today SSTableReader needs at minimum 3 files to load an sstable:
>  - -Data.db
>  - -CompressionInfo.db 
>  - -Index.db
> But during the scrub process the -Index.db file isn't actually necessary, unless there's
corruption in the -Data.db and we want to be able to skip over corrupted rows. Given that
there is still a fair chance that there's nothing wrong with the -Data.db file and we're just
missing the -Index.db file this patch addresses that situation.
> So the following patch makes it possible for the StandaloneScrubber (sstablescrub) to
recover sstables despite missing -Index.db files.
> This can happen from a catastrophic incident where data directories have been lost and/or
corrupted, or wiped and the backup not healthy. I'm aware that normally one depends on replicas
or snapshots to avoid such situations, but such catastrophic incidents do occur in the wild.
> I have not tested this patch against normal c* operations and all the other (more critical)
ways SSTableReader is used. i'll happily do that and add the needed units tests if people
see merit in accepting the patch.
> Otherwise the patch can live with the issue, in-case anyone else needs it. There's also
a cassandra distribution bundled with the patch [here|https://github.com/michaelsembwever/cassandra/releases/download/2.0.15-recover-sstables-without-indexdb/apache-cassandra-2.0.15-recover-sstables-without-indexdb.tar.gz]
to make life a little easier for anyone finding themselves in such a bad situation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message