cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <>
Subject [jira] Updated: (CASSANDRA-847) Make the reading half of compactions memory-efficient
Date Thu, 04 Mar 2010 04:37:29 GMT


Stu Hood updated CASSANDRA-847:

    Attachment: 0006-Add-support-for-compaction-of-super-CFs-and-some-tes.patch

This patchset applies something almost identical to the SSTableScanner interface from CASSANDRA-674
to our current file format, and uses the new interface for compactions.

Adds necessary datastructures from 674, and makes minor changes to other support classes

Implements the majority of SSTableScanner for our current file format, and adds shims to allow
various portions of the code to continue using iterators

Splits tests to allow us to test both the SSTableScanner api and direct FileDataInput access

Continues to flesh out the RowIndexedScanner implementation and tests, and separates SuperColumn
handling into a subclass for clarity

Adds a 'Scanner' interface which is more limited (and might apply to memtables?), implements
a FilteredScanner which filters slices in a wrapped Scanner, for use during AntiCompaction,
and adds the CompactionIterator implementation from 674

Adds support for compaction of super column families, which has one more bug to be ironed
out (but I wanted to post this set for feedback asap, since it is likely to change anyway)
and a few tests that failed to help me find the bug.

> Make the reading half of compactions memory-efficient
> -----------------------------------------------------
>                 Key: CASSANDRA-847
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Stu Hood
>            Priority: Critical
>             Fix For: 0.7
>         Attachments: 0001-Add-structures-that-were-important-to-the-SSTableSca.patch,
0002-Implement-most-of-the-new-SSTableScanner-interface.patch, 0003-Rename-RowIndexedReader-specific-test.patch,
0004-Improve-Scanner-tests-and-separate-SuperCF-handling-.patch, 0005-Add-Scanner-interface-and-a-Filtered-implementation-.patch,
> This issue is the next on the road to finally fixing CASSANDRA-16. To make compactions
memory efficient, we have to be able to perform the compaction process on the smallest possible
chunks that might intersect and contend one-another, meaning that we need a better abstraction
for reading from SSTables.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message