cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
Date Mon, 07 Sep 2015 01:08:47 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733187#comment-14733187
] 

Stefania commented on CASSANDRA-8630:
-------------------------------------

The following Coverity defects were reported after committing this, working on a fix:

{code}
*** CID 1322973:  FindBugs: Correctness  (FB.UR_UNINIT_READ_CALLED_FROM_SUPER_CONSTRUCTOR)
/src/java/org/apache/cassandra/io/util/RandomAccessReader.java: 83 in org.apache.cassandra.io.compress.CompressedRandomAccessReader.initializeBuffer()()
77             this.bufferType = builder.bufferType;
78
79             if (builder.bufferSize <= 0)
80                 throw new IllegalArgumentException("bufferSize must be positive");
81
82             if (builder.initializeBuffers)
>>>     CID 1322973:  FindBugs: Correctness  (FB.UR_UNINIT_READ_CALLED_FROM_SUPER_CONSTRUCTOR)
>>>     Call from superclass constructor here
83                 initializeBuffer();
84         }
85
86         protected int getBufferSize(Builder builder)
87         {
88             if (builder.limiter == null)

** CID 1322970:  FindBugs: Dodgy code  (FB.NS_DANGEROUS_NON_SHORT_CIRCUIT)
/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java: 140 in org.apache.cassandra.hints.ChecksummedDataInput.updateCrc()()


________________________________________________________________________________________________________
*** CID 1322970:  FindBugs: Dodgy code  (FB.NS_DANGEROUS_NON_SHORT_CIRCUIT)
/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java: 140 in org.apache.cassandra.hints.ChecksummedDataInput.updateCrc()()
134             super.reBuffer();
135             crcPosition = buffer.position();
136         }
137
138         private void updateCrc()
139         {
>>>     CID 1322970:  FindBugs: Dodgy code  (FB.NS_DANGEROUS_NON_SHORT_CIRCUIT)
>>>     Potentially dangerous use of non-short-circuit logic.
140             if (crcPosition == buffer.position() | crcUpdateDisabled)
141                 return;
142
143             assert crcPosition >= 0 && crcPosition < buffer.position();
144
145             ByteBuffer unprocessed = buffer.duplicate();

** CID 1322957:    (FORWARD_NULL)
{code}

> Faster sequential IO (on compaction, streaming, etc)
> ----------------------------------------------------
>
>                 Key: CASSANDRA-8630
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8630
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core, Tools
>            Reporter: Oleg Anastasyev
>            Assignee: Stefania
>              Labels: compaction, performance
>             Fix For: 3.x
>
>         Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, flight_recorder_001_files.tar.gz,
flight_recorder_002_files.tar.gz, mmaped_uncomp_hotspot.png
>
>
> When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot of CPU is
lost in calls to RAF's int read() and DataOutputStream's write(int).
> This is because default implementations of readShort,readLong, etc as well as their matching
write* are implemented with numerous calls of byte by byte read and write. 
> This makes a lot of syscalls as well.
> A quick microbench shows than just reimplementation of these methods in either way gives
8x speed increase.
> A patch attached implements RandomAccessReader.read<Type> and SequencialWriter.write<Type>
methods in more efficient way.
> I also eliminated some extra byte copies in CompositeType.split and ColumnNameHelper.maxComponents,
which were on my profiler's hotspot method list during tests.
> A stress tests on my laptop show that this patch makes compaction 25-30% faster  on uncompressed
sstables and 15% faster for compressed ones.
> A deployment to production shows much less CPU load for compaction. 
> (I attached a cpu load graph from one of our production, orange is niced CPU load - i.e.
compaction; yellow is user - i.e. not compaction related tasks)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message