cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
Date Sat, 24 Dec 2011 00:42:30 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175631#comment-13175631
] 

Pavel Yaskevich commented on CASSANDRA-3623:
--------------------------------------------

bq. We do get something like 50% better latencies by doing MMap'ed without copying the data.

But hot methods show the oposite, the main thing that hurts performance in the normal read
case is not memcopy but reader class initialization overhead.

bq. Snappy is 1.6% more because there isn't any thing else holding up or any other over head.

I don't get what do you mean here, can you please elaborate? Slower snappy execution on my
opinion could be caused by the additional expenses related to data mapping to the user-space
in the conditions of the migrating page cache (situation when dataset does not fit in the
page cache), mmap'ed I/O in that case makes kernel do more work comparing to syscalls (normal
I/O).

bq. Currently with this patch we dont have to copy any uncompressed data but the CRAR will
copy because we dont handle the DirectBB to snappy and that's made possible by using MMapped
IO.

Did you mean compressed instead of uncompressed here?
                
> use MMapedBuffer in CompressedSegmentedFile.getSegment
> ------------------------------------------------------
>
>                 Key: CASSANDRA-3623
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3623
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 1.1
>            Reporter: Vijay
>            Assignee: Vijay
>              Labels: compression
>             Fix For: 1.1
>
>         Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 0001-MMaped-Compression-segmented-file.patch,
0002-tests-for-MMaped-Compression-segmented-file-v2.patch
>
>
> CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the
MMap and hence a higher CPU on the nodes and higher latencies on reads. 
> This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
> // TODO refactor this to separate concept of "buffer to avoid lots of read() syscalls"
and "compression buffer"
> but i think a separate class for the Buffer will be better.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message