incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Wirt" <chris.w...@struq.com>
Subject RE: Repair session failed
Date Fri, 03 May 2013 09:02:46 GMT
Hi Aaron,

 

We're running 1.2.4, so with vNodes

 

We ran scrub but saw the issue again when repairing

 

nodetool status - 

 

Datacenter: DC01

=================

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

--  Address       Load       Tokens  Owns   Host ID
Rack

UN  10.70.48.23   35.16 GB   256     13.1%
4a7bc489-25af-4c20-80f8-499ffcb18e2d  RAC1

UN  10.70.6.79    30.04 GB   256     12.6%
98a1167f-cf75-4201-a454-695e0f7d2d72  RAC1

UN  10.70.6.78    41.94 GB   256     11.9%
62a418b5-3c38-4f66-874d-8138d6d565e5  RAC1

UN  10.70.47.66   54.79 GB   256     13.8%
ab564d16-4081-4866-b8ba-26461d9a93d7  RAC1

UN  10.70.6.91    46.96 GB   256     12.6%
2e1e7179-82e6-4ae6-b986-383acc9fc8a2  RAC1

UN  10.70.47.126  38.04 GB   256     11.8%
d4bed3b1-ffaf-4c68-b560-d270355c8c4b  RAC1

Datacenter: DC02

=================

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

--  Address       Load       Tokens  Owns   Host ID
Rack

UN  10.56.0.144   31.71 GB   256     12.0%
1860011e-fa7c-4ce1-ad6b-c8a38a5ddd02  RAC1

UN  10.56.0.140   86.28 GB   256     12.3%
f3fa985d-5056-4ddc-b146-d02432c3a86e  RAC1

 

 

Thanks,

 

Chris

 

 

From: aaron morton [mailto:aaron@thelastpickle.com] 
Sent: 02 May 2013 19:31
To: user@cassandra.apache.org
Subject: Re: Repair session failed

 

Hold off on running scrub (but yes it's an online operation). This is an
issue with the token ranges. 

 

What version are you using ? 

Are you using vNodes ?

Can you share the output of nodetool ring (if no vnodes) or nodetool status
(if using vnodes) ?

 

Cheers

 

-----------------

Aaron Morton

Freelance Cassandra Consultant

New Zealand

 

@aaronmorton

http://www.thelastpickle.com

 

On 2/05/2013, at 3:08 AM, Haithem Jarraya <haithem.jarraya@struq.com> wrote:





Can I run scrub while the node is in the ring and receiving writes?

Or I should disable thrift before?

 

On 1 May 2013 15:52, <moshe.kranc@barclays.com> wrote:

Sounds like a job for "nodetool scrub", which rewrites the SStable rows in
the correct order. After the scrub, nodetool repair should succeed.

 

From: Haithem Jarraya [mailto:haithem.jarraya@struq.com] 
Sent: Wednesday, May 01, 2013 5:46 PM
To: user@cassandra.apache.org
Subject: Repair session failed

 

Hi, 

 

I am seeing this error message during repair,

 

 INFO [AntiEntropyStage:1] 2013-05-01 14:30:54,300 AntiEntropyService.java
(line 764) [repair #ed104480-b26a-11e2-af9b-05179fa66b76] mycolumnfamily is
fully synced (1 remaining column family to sync for this session)

ERROR [Thread-12725] 2013-05-01 14:30:54,304 StorageService.java (line 2420)
Repair session failed:

java.lang.IllegalArgumentException: Requested range intersects a local range
but is not fully contained in one; this would lead to imprecise repair

        at
org.apache.cassandra.service.AntiEntropyService.getNeighbors(AntiEntropyServ
ice.java:175)

        at
org.apache.cassandra.service.AntiEntropyService$RepairSession.<init>(AntiEnt
ropyService.java:621)

        at
org.apache.cassandra.service.AntiEntropyService$RepairSession.<init>(AntiEnt
ropyService.java:610)

        at
org.apache.cassandra.service.AntiEntropyService.submitRepairSession(AntiEntr
opyService.java:127)

        at
org.apache.cassandra.service.StorageService.forceTableRepair(StorageService.
java:2480)

        at
org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.jav
a:2416)

        at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)

        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)

        at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)

        at java.lang.Thread.run(Thread.java:662)

 

 

what does it mean imprecise repair?

Is it maybe because I went over the gcgrade period?

What you do if you go over that period?

Any hint will be valuable, 

Also I noticed when I run a repair on different node, I see a message like
this

 

[2013-05-01 14:30:54,305] Starting repair command #5, repairing 1120 ranges
for keyspace struqrealtime

 

I have couple of questions, why I have repair command #5?

And why the ranges values changes from one node to another?

 

 

Many Thanks,

 

H

_______________________________________________

This message is for information purposes only, it is not a recommendation,
advice, offer or solicitation to buy or sell a product or service nor an
official confirmation of any transaction. It is directed at persons who are
professionals and is not intended for retail customer use. Intended for
recipient only. This message is subject to the terms at:
www.barclays.com/emaildisclaimer.

For important disclosures, please see:
www.barclays.com/salesandtradingdisclaimer regarding market commentary from
Barclays Sales and/or Trading, who are active market participants; and in
respect of Barclays Research, including disclosures relating to specific
issuers, please see http://publicresearch.barclays.com
<http://publicresearch.barclays.com/> .

_______________________________________________

 

 


Mime
View raw message