cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <>
Subject [jira] Updated: (CASSANDRA-847) Make the reading half of compactions memory-efficient
Date Mon, 29 Mar 2010 04:42:27 GMT


Stu Hood updated CASSANDRA-847:

    Attachment: compaction-bench-trunk.txt

Starting with step 0. from jbellis's suggested order, I implemented a compaction benchmark,
and tested trunk and a rebased version of this patch. As expected, this patch is significantly
faster for wide rows and slightly slower for narrow rows, but I'm sure that the performance
can be improved by cleaning up the mergesort in SliceBuffer.merge().

The reason I bring this up now is that I began work on 767, and realized how painful it was
going to be to make format changes before we had settled on an interface to replace FileDataInput/SSTableScanner.
We need to remove/improve those interfaces before we go about making sweeping changes like
the String->byte[] refactor.

I think that this patch begins the 674 refactor in the correct place, so rather than starting
over and losing more time, I would love to be able to clean up this patch and remove any structures
that you guys think are excessive. If I can squash this down into much clearer patches and
remove more of the speculative code, what are the chances of getting it committed?

> Make the reading half of compactions memory-efficient
> -----------------------------------------------------
>                 Key: CASSANDRA-847
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Stu Hood
>            Priority: Critical
>             Fix For: 0.7
>         Attachments: 0001-Add-structures-that-were-important-to-the-SSTableSca.patch,
0002-Implement-most-of-the-new-SSTableScanner-interface.patch, 0003-Rename-RowIndexedReader-specific-test.patch,
0004-Improve-Scanner-tests-and-separate-SuperCF-handling-.patch, 0005-Add-Scanner-interface-and-a-Filtered-implementation-.patch,
0006-Add-support-for-compaction-of-super-CFs-and-some-tes.patch, 0007-Remove-ColumnKey-bloom-filter-maintenance.patch,
0008-Make-Scanner-extend-Iterator-again.patch, 0009-Make-CompactionIterator-a-ReducingIterator-subclass-.patch,
0010-Alternative-to-ReducingIterator-that-can-return-mult.patch, compaction-bench-847.txt,
compaction-bench-trunk.txt, compaction-bench.patch
> This issue is the next on the road to finally fixing CASSANDRA-16. To make compactions
memory efficient, we have to be able to perform the compaction process on the smallest possible
chunks that might intersect and contend one-another, meaning that we need a better abstraction
for reading from SSTables.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message