cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arend-Jan Wijtzes <ajwyt...@wise-guys.nl>
Subject endless hinted handoff with 1.1
Date Mon, 21 May 2012 08:36:40 GMT
Hi,

We are running a small test cluster and recently installed Cassandra 1.1 and
started with a new clean database. We keep seeing these messages in the log 
on just one of our nodes:

INFO [HintedHandoff:1] 2012-05-21 09:49:56,757 HintedHandOffManager.java (line 294) Started
hinted handoff for token: 85070591730234615865843651857942052864 with IP: /10.0.0.73
 INFO [HintedHandoff:1] 2012-05-21 09:49:56,775 HintedHandOffManager.java (line 382) Finished
hinted handoff of 0 rows to endpoint /10.0.0.73
 INFO [HintedHandoff:1] 2012-05-21 09:59:56,756 HintedHandOffManager.java (line 294) Started
hinted handoff for token: 42535295865117307932921825928971026432 with IP: /10.0.0.69
 INFO [HintedHandoff:1] 2012-05-21 09:59:56,757 HintedHandOffManager.java (line 382) Finished
hinted handoff of 0 rows to endpoint /10.0.0.69
 INFO [HintedHandoff:1] 2012-05-21 09:59:56,757 HintedHandOffManager.java (line 294) Started
hinted handoff for token: 85070591730234615865843651857942052864 with IP: /10.0.0.73
 INFO [HintedHandoff:1] 2012-05-21 09:59:56,775 HintedHandOffManager.java (line 382) Finished
hinted handoff of 0 rows to endpoint /10.0.0.73
 INFO [HintedHandoff:1] 2012-05-21 10:09:56,757 HintedHandOffManager.java (line 294) Started
hinted handoff for token: 42535295865117307932921825928971026432 with IP: /10.0.0.69
 INFO [HintedHandoff:1] 2012-05-21 10:09:56,758 HintedHandOffManager.java (line 382) Finished
hinted handoff of 0 rows to endpoint /10.0.0.69
 INFO [HintedHandoff:1] 2012-05-21 10:09:56,758 HintedHandOffManager.java (line 294) Started
hinted handoff for token: 85070591730234615865843651857942052864 with IP: /10.0.0.73
 INFO [HintedHandoff:1] 2012-05-21 10:09:56,879 HintedHandOffManager.java (line 382) Finished
hinted handoff of 0 rows to endpoint /10.0.0.73


All four nodes are up:

-bash-4.1$ nodetool ring -h localhost
Note: Ownership information does not include topology, please specify a keyspace. 
Address         DC          Rack        Status State   Load            Owns              
 Token                                       
                                                                                         
 127605887595351923798765477786913079296     
10.0.0.65       datacenter1 rack1       Up     Normal  244.41 MB       25.00%            
 0                                           
10.0.0.69       datacenter1 rack1       Up     Normal  155.39 MB       25.00%            
 42535295865117307932921825928971026432      
10.0.0.73       datacenter1 rack1       Up     Normal  220.42 MB       25.00%            
 85070591730234615865843651857942052864      
10.0.0.77       datacenter1 rack1       Up     Normal  296.14 MB       25.00%            
 127605887595351923798765477786913079296     


This has been going on for days. Note that it's just two key's in the log
that keep repeating. No recent messages about HintedHandOff in the logs 
on the other nodes.

Let me know if you need more info.

Arend-Jan

-- 
Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl

Mime
View raw message