cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "T Jake Luciani (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9766) Bootstrap outgoing streaming speeds are much slower than during repair
Date Wed, 13 Apr 2016 14:26:25 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239330#comment-15239330
] 

T Jake Luciani commented on CASSANDRA-9766:
-------------------------------------------

 I've taken a look at the streaming path and have been able to improve Streaming performance
25% with the following:

* CompressedStreamReader had only implemented reading a single byte at a time. I added the
read(byte[], int, int) method.

* When writing to sstable, rather than calculate size of row, write the size then write the
row (which causes 2x the cpu). Serialize the row to memory then write size of memory buffer
and copy buffer to disk.

* Added object recycling of the largest garbage sources. Namely, BTreeSearchIterator, and
DataOutputBuffer (for above fix). There are still a few more places recycling would help/
like the Object[] in BTree.Builder

* Changed all ThreadLocals to use FastThreadLocal from netty, and subsequently adding FastThreadLocalThreads
for all internal threads.

There are still more things todo here, like we generate tons of garbage boxing types for StreamingHistogram.

I've added a long test to stream and compact a large sstable.

Branch is https://github.com/tjake/cassandra/tree/faster-streaming

3.5 test:
Finished Streaming in 25 seconds: 18.92 Mb/sec

branch test:
Finished Streaming in 19 seconds: 25.04 Mb/sec

> Bootstrap outgoing streaming speeds are much slower than during repair
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-9766
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9766
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>         Environment: Cassandra 2.1.2. more details in the pdf attached 
>            Reporter: Alexei K
>            Assignee: T Jake Luciani
>              Labels: performance
>             Fix For: 2.1.x
>
>         Attachments: problem.pdf
>
>
> I have a cluster in Amazon cloud , its described in detail in the attachment. What I've
noticed is that we during bootstrap we never go above 12MB/sec transmission speeds and also
those speeds flat line almost like we're hitting some sort of a limit ( this remains true
for other tests that I've ran) however during the repair we see much higher,variable sending
rates. I've provided network charts in the attachment as well . Is there an explanation for
this? Is something wrong with my configuration, or is it a possible bug?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message