incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Coli <rc...@eventbrite.com>
Subject Re: Data tombstoned during bulk loading 1.2.10 -> 2.0.3
Date Tue, 04 Feb 2014 18:39:56 GMT
On Tue, Feb 4, 2014 at 12:21 AM, olek.stasiak@gmail.com <
olek.stasiak@gmail.com> wrote:

> I don't know what is the real cause of my problem. We are still guessing.
> All operations I have done one cluster are described on timeline:
> 1.1.7-> 1.2.10 -> upgradesstable -> 2.0.2 -> normal operations ->2.0.3
> -> normal operations -> now
> normal operations means reads/writes/repairs.
> Could you please, describe briefly how to recover data? I have a
> problem with scenario described under link:
>
> http://thelastpickle.com/blog/2011/12/15/Anatomy-of-a-Cassandra-Partition.html,
> I can't apply this solution to my case.
>

I think your only option is the following :

1) determine which SSTables contain rows have doomstones (tombstones from
the far future)
2) determine whether these tombstones mask a live or dead version of the
row, by looking at other row fragments
3) dump/filter/re-write all your data via some method, probably
sstable2json/json2sstable
4) load the corrected sstables by starting a node with the sstables in the
data directory

I understand you have a lot of data, but I am pretty sure there is no way
for you to fix it within Cassandra. Perhaps ask for advice on the JIRA
ticket mentioned upthread if this answer is not sufficient?

=Rob

Mime
View raw message