cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anubhav Kale (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.
Date Tue, 21 Jun 2016 17:27:58 GMT


Anubhav Kale commented on CASSANDRA-4663:

I ran some more tests on the original code and change with multiple sockets, and confirmed
that the end-to-end time we see during streaming is a direct function of how long it takes
for the sender to send bytes through (meaning sender is the only "slow" entity which makes
the problem somewhat tangible).

Then, I tested sending multiple files in parallel through some hacks, but as I was expecting
it does not yield much improvements mainly because {{WritableByteChannel}} is a blocking channel
across threads. 

>From docs, "Only one write operation upon a writable channel may be in progress at any
given time. If one thread initiates a write operation upon a channel then any other thread
that attempts to initiate another write operation will block until the first operation is

We would need to move to {{AsynchronousSocketChannel}} to get true parallelism (which obviously
is a deeper change - not impossible though).

> Streaming sends one file at a time serially. 
> ---------------------------------------------
>                 Key: CASSANDRA-4663
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: sankalp kohli
>            Priority: Minor
> This is not fast enough when someone is using SSD and may be 10G link. We should try
to create multiple connections and send multiple files in parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 

This message was sent by Atlassian JIRA

View raw message