cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joshua McKenzie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-8709) Convert SequentialWriter from using RandomAccessFile to nio channel
Date Fri, 20 Feb 2015 01:18:11 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328414#comment-14328414
] 

Joshua McKenzie commented on CASSANDRA-8709:
--------------------------------------------

Initial pass at this available [here|https://github.com/josh-mckenzie/cassandra/compare/8709].

SequentialWriter, CompressedSequentialWriter, and ChecksummedSequentialWriter all now rely
on a ByteBuffer instead of byte[] and CSW now uses a WrappedByteBuffer for compression. Old
byte[] based compression methods are removed in favor of Bytebuffer based. Unit tests updated,
some new unit tests added, and all unit tests pass on both linux and Windows (including SSTRW
tests on Windows!). Ran a flight recorder for mmap idx, 20M ops, write for both 2.1 and 8709
- GC results and allocation indicate a 2-2.5% decrease in allocations and GC count on this
patch vs trunk though I suspect that's within the margin of error on the test. I'd attach
those .jfr but it was on a linux vm so they're pretty huge as throughput was low.  I'm not
seeing any performance regressions either in terms of memory usage or throughput on writes.

Some closing thoughts:
* This further couples us to Adler32 in ChecksumWriter (necessary for the direct checksum
update reflection magic - thanks [~tjake])
* I think that reading from the FileChannel into a Bytebuffer for crc confirmation means there's
a chance we could false-negative on the crc check if the previous value happens to match the
next write. Starting w/a fresh ByteBuffer on each calculation doesn't make any difference
as straight byte 0's is just as likely to show up as any other calculation result.

> Convert SequentialWriter from using RandomAccessFile to nio channel
> -------------------------------------------------------------------
>
>                 Key: CASSANDRA-8709
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8709
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Joshua McKenzie
>            Assignee: Joshua McKenzie
>              Labels: Windows
>             Fix For: 3.0
>
>
> For non-mmap'ed I/O on Windows, using nio channels will give us substantially more flexibility
w/regards to renaming and moving files around while writing them.  This change in conjunction
with CASSANDRA-4050 should allow us to remove the Windows bypass code in SSTableRewriter for
non-memory-mapped I/O.
> In general, migrating from instances of RandomAccessFile to nio channels will help make
Windows and linux behavior more consistent.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message