cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anubhav Kale (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.
Date Tue, 21 Jun 2016 17:27:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342248#comment-15342248
] 

Anubhav Kale commented on CASSANDRA-4663:
-----------------------------------------

I ran some more tests on the original code and change with multiple sockets, and confirmed
that the end-to-end time we see during streaming is a direct function of how long it takes
for the sender to send bytes through (meaning sender is the only "slow" entity which makes
the problem somewhat tangible).

Then, I tested sending multiple files in parallel through some hacks, but as I was expecting
it does not yield much improvements mainly because {{WritableByteChannel}} is a blocking channel
across threads. 

>From docs, "Only one write operation upon a writable channel may be in progress at any
given time. If one thread initiates a write operation upon a channel then any other thread
that attempts to initiate another write operation will block until the first operation is
complete."

We would need to move to {{AsynchronousSocketChannel}} to get true parallelism (which obviously
is a deeper change - not impossible though).


> Streaming sends one file at a time serially. 
> ---------------------------------------------
>
>                 Key: CASSANDRA-4663
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: sankalp kohli
>            Priority: Minor
>
> This is not fast enough when someone is using SSD and may be 10G link. We should try
to create multiple connections and send multiple files in parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message