cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Morishita (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-4726) Streaming sstables during repair is limited by compression
Date Wed, 26 Sep 2012 23:01:07 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yuki Morishita resolved CASSANDRA-4726.
---------------------------------------

       Resolution: Duplicate
    Fix Version/s: 1.2.0

see CASSANDRA-4297
                
> Streaming sstables during repair is limited by compression
> ----------------------------------------------------------
>
>                 Key: CASSANDRA-4726
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4726
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.5
>            Reporter: Oleg Kibirev
>             Fix For: 1.2.0
>
>
> In our benchmarks of node rebuilds, streaming SSTables is limited by sender uncompressing
them using snappy and compressing using lzf. The receiver has to reverse this process and
is likewise bottlenecked in compression code. We are therefore only able to utilize less than
1/2 of receiver's 1gbps network link in a test with one receiver and two senders.
> If sstable is already compressed using snappy, it should be just sent to receiver as
is. 
> Here are hprof fragments excluding socket accepts and reads by idle threads:
> Sender:
>   4  3.14% 95.51%    6128 302816 com.ning.compress.lzf.ChunkEncoder.tryCompress
>    5  1.93% 97.44%    3761 302815 java.net.SocketOutputStream.socketWrite0
>    6  1.27% 98.71%    2477 301463 java.util.zip.CRC32.updateBytes
>    7  0.41% 99.12%     798 302822 com.ning.compress.lzf.ChunkEncoder.tryCompress
>    8  0.31% 99.43%     602 301465 org.xerial.snappy.SnappyNative.rawUncompress
>    9  0.18% 99.60%     349 302820 java.net.SocketInputStream.socketRead0
>   10  0.14% 99.75%     282 302817 java.net.SocketOutputStream.socketWrite0
>   11  0.14% 99.89%     276 301468 java.io.FileInputStream.readBytes
>   12  0.03% 99.92%      52 301929 sun.nio.ch.FileDispatcher.preClose0
> Receiver:
>   4  7.15% 84.63%    2782 300893 org.xerial.snappy.SnappyNative.rawCompress
>    5  3.16% 87.79%    1232 300895 java.util.zip.CRC32.updateBytes
>    6  2.83% 90.62%    1101 300896 com.ning.compress.lzf.LZFDecoder.decompressChunk
>    7  1.50% 92.12%     583 300916 java.io.DataInputStream.readFully
>    8  1.46% 93.57%     568 300921 java.io.FileDescriptor.sync
>    9  1.16% 94.73%     451 300910 java.io.RandomAccessFile.writeBytes
>   10  0.47% 95.20%     184 300949 sun.nio.ch.NativeThread.current
>   11  0.45% 95.66%     177 300926 java.io.RandomAccessFile.getFilePointer
>   12  0.31% 95.97%     121 300935 org.apache.cassandra.io.util.SequentialWriter.write

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message