cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Koblas <da...@koblas.com>
Subject Re: Repair Failing due to bad network
Date Fri, 12 Oct 2012 15:48:09 GMT
Jim,

Great idea - though it doesn't look like it's in 1.1.3 (which is what 
I'm running).

My lame idea of the morning is that I'm going to just read the whole 
keyspace with QUORUM reads to force read repairs - the unfortunate truth 
is that this is about 2B reads...

--david

On 10/11/12 4:51 PM, Jim Cistaro wrote:
> I am not aware of any built-in mechanism for retrying repairs.  I believe
> you will have to build that into your process.
>
> As for reducing the time of each repair command to fit in your windows:
>
> If you have multiple reasonable size column families, and are not already
> doing this, one approach might be to do repairs on a per cf basis.  This
> will break your repairs up into smaller chunks that might fit in the
> window.
>
> If you are not doing -pr (primary range), using that on each node causes
> the repair command to only repair the primary range on the node (not the
> ones it is replicating).
>
> Depending on your version, there is also
> https://issues.apache.org/jira/browse/CASSANDRA-3912 which might help you
> - but I have no experience using this feature.
>
> jc
>
> On 10/11/12 4:09 PM, "David Koblas" <david@koblas.com> wrote:
>
>> I'm trying to bring up a new Datacenter - while I probably could have
>> brought things up in another way I've now got a DC that has a ready
>> Cassandra with keys allocated.  The problem is that I cannot get a
>> repair to complete due since it appears that some part of my network
>> decides to restart all connections twice a day (6am and 2pm - ok 5
>> minutes before).
>>
>> So when I start a repair job, it usually get's a ways into things before
>> one of the nodes goes DOWN, then back up.  What I don't see is the
>> repair restarting, it just stops.
>>
>> Is there a workaround for this case, or is there something else I could
>> be doing?
>>
>> --david
>>


Mime
View raw message