cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <>
Subject [jira] Commented: (CASSANDRA-1117) Clean up MMAP support
Date Tue, 25 May 2010 21:01:32 GMT


Stu Hood commented on CASSANDRA-1117:

> this patch adds complexity (and poor performance - creating BRAFs is not free) to the
non-mmap'd case by creating BRAF segments
This patch isn't supposed to create any more BRAFs than trunk does: in the non-mmap case,
a SegmentedFile with one "null" segment should be created, which should result in a BRAF being
created for every getSegment/getFileDataInput call. I think this is equivalent to the current

> It looks like a single row has to fit in 2GB here, which isn't going to be acceptable.
The segment building support (look at SegmentedFile.addPotentialBoundary) attempts to create
segments that are just barely smaller than the 2GB, but failing that, it will create a segment
larger than 2GB. The segment which is larger than 2GB will be a 'null' segment, meaning that
getFileDataInput calls for that segment will use a BRAF.

> I'd say looking up index segment by navigablemap instead of % is the cause of most of
the lost performance (in mmap mode).
Agreed... it may be possible to optimize this lookup somehow, but I think it might be worth
eating the cost.

> Clean up MMAP support
> ---------------------
>                 Key: CASSANDRA-1117
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Stu Hood
>            Assignee: Gary Dusbabek
>             Fix For: 0.7
>         Attachments: 0001-Use-factory-functions-for-RowIndexedReader.patch, 0002-Add-SegmentedFile-to-abstract-opening-FileDataInputs.patch,
0003-Replace-mmap-file-abstraction-with-SegmentedFile.patch, 0004-Rename-SSTableReaderTest-to-SegmentedFileTest.patch,
> Awareness of MMAP is currently embedded into the SSTableReader implementation and IndexSummary.
A good number of bugs experienced recently have been due to this lack of separation, so it
is ripe for abstraction. Additionally, the current implementation does not provide a good
method for iterating over the segments of a file, which is useful for range queries, and lays
more stable groundwork for #998.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message