cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gábor Auth <auth.ga...@gmail.com>
Subject Slow repair
Date Wed, 15 Mar 2017 09:50:22 GMT
Hi,

We are working with a two DCs Cassandra cluster (EU and US), so that the
distance is over 160 ms between them. I've added a new DC to this cluster,
modified the keyspace's replication factor and trying to rebalance it with
repair but the repair is very slow (over 10-15 minutes per node per
keyspace with ~40 column families). Is it normal with this network latency
or something wrong with the cluster or the network connection? :)

[2017-03-15 05:52:38,255] Starting repair command #4, repairing keyspace
test20151222 with repair options (parallelism: parallel, primary range:
true, incremental: false, job threads: 1, ColumnFamilies: [], dataCenters:
[], hosts: [], # of ranges: 32)
[2017-03-15 05:54:11,913] Repair session
988bd850-0943-11e7-9c1f-f5ba092c6aea for range
[(-3328182031191101706,-3263206086630594139],
(-449681117114180865,-426983008087217811],
(-4940101276128910421,-4726878962587262390],
(-4999008077542282524,-4940101276128910421]] finished (progress: 11%)
[2017-03-15 05:55:39,721] Repair session
9a6fda92-0943-11e7-9c1f-f5ba092c6aea for range
[(7538662821591320245,7564364667721298414],
(8095771383100385537,8112071444788258953],
(-1625703837190283897,-1600176580612824092],
(-1075557915997532230,-1072724867906442440], (-9152
563942239372475,-9123254980705325471],
(7485905313674392326,7513617239634230698]] finished (progress: 14%)
[2017-03-15 05:57:05,718] Repair session
9de181b1-0943-11e7-9c1f-f5ba092c6aea for range
[(-6471953894734787784,-6420063839816736750],
(1372322727565611879,1480899944406172322],
(1176263633569625668,1177285361971054591],
(440549646067640682,491840653569315468], (-43128299
75221321282,-4177428401237878410]] finished (progress: 17%)
[2017-03-15 05:58:39,997] Repair session
a18bc500-0943-11e7-9c1f-f5ba092c6aea for range
[(5327651902976749177,5359189884199963589],
(-5362946313988105342,-5348008210198062914],
(-5756557262823877856,-5652851311492822149],
(-5400778420101537991,-5362946313988105342], (668
2536072120412021,6904193483670147322]] finished (progress: 20%)
[2017-03-15 05:59:11,791] Repair session
a44f2ac2-0943-11e7-9c1f-f5ba092c6aea for range
[(952873612468870228,1042958763135655298],
(558544893991295379,572114658167804730]] finished (progress: 22%)
[2017-03-15 05:59:56,197] Repair session
a5e13c71-0943-11e7-9c1f-f5ba092c6aea for range
[(1914238614647876002,1961526714897144472],
(3610056520286573718,3619622957324752442],
(-3506227577233676363,-3504718440405535976],
(-4120686433235827731,-4098515820338981500], (56515
94158011135924,5668698324546997949]] finished (progress: 25%)
[2017-03-15 06:00:45,610] Repair session
a897a9e1-0943-11e7-9c1f-f5ba092c6aea for range
[(-9007733666337543056,-8979974976044921941]] finished (progress: 28%)
[2017-03-15 06:01:58,826] Repair session
a927b4e1-0943-11e7-9c1f-f5ba092c6aea for range
[(3599745202434925817,3608662806723095677],
(3390003128426746316,3391135639180043521],
(3391135639180043521,3529019003015169892]] finished (progress: 31%)
[2017-03-15 06:03:15,440] Repair session
aae06160-0943-11e7-9c1f-f5ba092c6aea for range
[(-7542303048667795773,-7300899534947316960]] finished (progress: 34%)
[2017-03-15 06:03:17,786] Repair completed successfully
[2017-03-15 06:03:17,787] Repair command #4 finished in 10 minutes 39
seconds

Bye,
Gábor Auth

Mime
View raw message