The point is that there is no way, afaik, to limit the speed of these Hinted Handoff since it's not a stream like repair or bootstrap, no way either to keep the node out of the ring during the time it is receiving hints since hints and "normal" traffic both go through gossip protocol on port 7000.

How to avoid this Hinted Handoff flood on returning nodes ?

Alain


2013/7/4 Alain RODRIGUEZ <arodrime@gmail.com>
Hi,

Using C*1.2.2 12 EC2 xLarge cluster.

When I restart a node, if it spend a few minutes down, when I bring it up, all the cpu are blocked at 100%, even once compactions are disabled, inducing a very big and intolerable latency in my app. I suspect Hinted Handoff to be the cause of this. disabling gossip fix the problem, enabling it again brings the latency back (with a lot of gc, dropped messages...).

Is there a way to disable HH ? Are they responsible for this issue ?

I currently have this node down, any fast insight would be appreciated.

Alain