cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay (Commented) (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-3623) use MMapedBuffer in CompressedSegmentedFile.getSegment
Date Sat, 24 Dec 2011 00:16:30 GMT


Vijay commented on CASSANDRA-3623:

Regarding duplicates i was thinking of Creating duplicates in CMSF and having a helper function
to track it.

Regarding Hot Reads: (I tried before and you have to access the FD and caching the initialized
object didn't help), We do get something like 50% better latencies by doing MMap'ed without
copying the data. Snappy is 1.6% more because there isn't any thing else holding up or any
other over head. 

Currently with this patch we dont have to copy any uncompressed data but the CRAR will copy
because we dont handle the DirectBB to snappy and that's made possible by using MMapped IO.
> use MMapedBuffer in CompressedSegmentedFile.getSegment
> ------------------------------------------------------
>                 Key: CASSANDRA-3623
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 1.1
>            Reporter: Vijay
>            Assignee: Vijay
>              Labels: compression
>             Fix For: 1.1
>         Attachments: 0001-MMaped-Compression-segmented-file-v2.patch, 0001-MMaped-Compression-segmented-file.patch,
> CompressedSegmentedFile.getSegment seem to open a new file and doesnt seem to use the
MMap and hence a higher CPU on the nodes and higher latencies on reads. 
> This ticket is to implement the TODO mentioned in CompressedRandomAccessReader
> // TODO refactor this to separate concept of "buffer to avoid lots of read() syscalls"
and "compression buffer"
> but i think a separate class for the Buffer will be better.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message