cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <>
Subject Network / GC / Latency spike
Date Mon, 31 Aug 2015 13:48:03 GMT

Running a 2.0.16 C* on AWS (private VPC, 2 DC).

I am facing an issue on our EU DC where I have a network burst (alongside
with GC and latency increase).

My first thought was a sudden application burst, though, I see no
corresponding evolution on reads / write or even CPU.

So I thought that this might come from the node themselves as IN almost
equal OUT Network. I tried lowering stream throughput on the whole DC to 1
Mbps, with ~30 nodes --> 30 Mbps --> ~4 MB/s max. My network went a lot
higher about 30 M in both sides (see screenshots attached).

I have tried to use iftop to see where this network is headed too, but I
was not able to do it because burst are very shorts.

So, questions are:

- Did someone experienced something similar already ? If so, any clue would
be appreciated :).
- How can I know (monitor, capture) where this big amount of network is
headed to or due to ?
- Am I right trying to figure out what this network is or should I follow
an other lead ?

Notes: I also noticed that CPU does not spike nor does R&W, but disk reads
also spikes !



View raw message