cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anuj Wadehra <>
Subject Re: MUTATION messages were dropped in last 5000 ms for cross node timeout
Date Thu, 20 Jul 2017 17:16:36 GMT
Hi Asad, 
You can do following things:
1.Increase memtable_flush_writers especially if you have a write heavy load. 
2.Make sure there are no big gc pauses on your nodes. If yes,  go for heap tuning. 

Please let us know whether above measures fixed your problem or not. 


Sent from Yahoo Mail on Android 
  On Thu, 20 Jul 2017 at 20:57, ZAIDI, ASAD A<> wrote:    <!--#yiv1235039646
_filtered #yiv1235039646 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered
#yiv1235039646 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv1235039646 #yiv1235039646
p.yiv1235039646MsoNormal, #yiv1235039646 li.yiv1235039646MsoNormal, #yiv1235039646 div.yiv1235039646MsoNormal
{margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", sans-serif;}#yiv1235039646
a:link, #yiv1235039646 span.yiv1235039646MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv1235039646
a:visited, #yiv1235039646 span.yiv1235039646MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv1235039646
span.yiv1235039646EmailStyle17 {font-family:"Calibri", sans-serif;color:windowtext;}#yiv1235039646
.yiv1235039646MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv1235039646
{margin:1.0in 1.0in 1.0in 1.0in;}#yiv1235039646 div.yiv1235039646WordSection1 {}-->
Hello Folks –
I’m using apache-cassandra 2.2.8.
I see many messages like below in my system.log file. In Cassandra.yaml file [ cross_node_timeout:
true] is set and NTP server is also running correcting clock drift on 16node cluster. I do
not see pending or blocked HintedHandoff  in tpstats output though there are bunch of MUTATIONS
dropped observed.
<start timeout message >
INFO  [ScheduledTasks:1] 2017-07-20 08:02:52,511 - MUTATION messages
were dropped in last 5000 ms: 822 for internal timeout and 2152 for cross node timeout
<end timeout message >
I’m seeking help here if you please let me know what I need to check in order to address
these cross node timeouts.
Thank you,

View raw message