cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steinmaurer, Thomas" <>
Subject RE: Cassandra 2.1.18 - Question on stream/bootstrap throughput
Date Tue, 22 Oct 2019 15:10:17 GMT
Hi Alex,

Increased streaming throughput has been set on the existing nodes only, cause it is meant
to limit outgoing traffic only, right? At least when judging from the name, reading the documentation

Increased compaction throughput on all nodes, although my understanding is that it would be
necessary only on the joining node to catchup with compacting received SSTables.

We really see no resource (CPU, NW and disk) being somehow maxed out on any node, which would
explain the limit in the area of the new node receiving data at ~ 180-200 Mbit/s.

Thanks again,

From: Oleksandr Shulgin <>
Sent: Dienstag, 22. Oktober 2019 16:35
To: User <>
Subject: Re: Cassandra 2.1.18 - Question on stream/bootstrap throughput

On Tue, Oct 22, 2019 at 12:47 PM Steinmaurer, Thomas <<>>

using 2.1.8, 3 nodes (m4.10xlarge, ESB SSD-based), vnodes=256, RF=3, we are trying to add
a 4th node.

The two options to my knowledge, mainly affecting throughput, namely stream output and compaction
throttling has been set to very high values (e.g. stream output = 800 Mbit/s resp. compaction
throughput = 500 Mbyte/s) or even set to 0 (unthrottled) in cassandra.yaml + process restart.
In both scenarios (throttling with high values vs. unthrottled), the 4th node is streaming
from one node capped ~ 180-200Mbit/s, according to our SFM.

The nodes have plenty of resources available (10Gbit, disk io/iops), also confirmed by e.g.
iperf in regard to NW throughput and write to / read from disk in the area of 200 MByte/s.

Are there any other known throughput / bootstrap limitations, which basically outrule above

Hi Thomas,

Assuming you have 3 Availability Zones and you are adding the new node to one of the zones
where you already have a node running, it is expected that it only streams from that node
(its local rack).

Have you increased the streaming throughput on the node it streams from or only on the new
node?  The limit applies to the source node as well.  You can change it online w/o the need
to restart using nodetool command.

Have you checked if the new node is not CPU-bound?  It's unlikely though due to big instance
type and only one node to stream from, more relevant for scenarios when streaming from a lot
of nodes.


The contents of this e-mail are intended for the named addressee only. It contains information
that may be confidential. Unless you are the named addressee or an authorized designee, you
may not copy or use it, or disclose it to anyone else. If you received it in error please
notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN
91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria,
Freistädterstraße 313
View raw message