incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Lebresne <>
Subject Re: Why data tripled in size after repair?
Date Thu, 27 Sep 2012 18:26:20 GMT
> I see. It explains why I get 85G + 85G instead of 90G. But after next
> repair I have six extra files 75G each,
> how is it possible?

Maybe you've run repair on other nodes? Basically repair is a fairly
blind process. If it consider that a given range (and by range I mean
here the ones that repair hashes, so they're suppose to be small
ranges) is inconsistent between two peers, it doesn't know which peers
is up to date (and in fact it cannot, it is possible that no node is
more up to date than the other on the whole range, even if said range
is small, at least in theory).

> It looks like repair is done per sstable, not CF. Is it possible?

No, repair is done on the CF, not on individual sstables.

> Does Merklee tree calculation algorithm use sstables flushed on hard drive or it uses
mem tables also?

It triggers a flush, wait for it, and then use the on-disk sstables.
So in theory, since the order to flush is sent to each replica at the
same time, all the replica will trigger the flush in a very short
interval and thus consider data set that only from that short
interval. So in a cluster with a high write load, it is expected that
a bit of the inconsistence is due to that short interval, but it
should be relatively negligible. That's the theory at least. But
clearly your case (ordered partitioner with intensive update covering
the whole range) does maximize that inconsistency. Still, it shouldn't
be that dramatic.

However your use of the ordered partitionner might not be
insignificant as it's much less used and repair does have a few
specific bits for it. Do you mind opening a ticket on jira with a
summary of your configuration/problem? I'll look if I can spot
something wrong relating to the order partitioner but in any that'll
be simpler to track what's wrong there.


View raw message