cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <>
Subject [jira] Commented: (CASSANDRA-1472) Add bitmap secondary indexes
Date Sun, 16 Jan 2011 22:21:44 GMT


Stu Hood commented on CASSANDRA-1472:

> What do you think?
I think that implementing a compressible block based file format is a non-trivial task, and
that before we commit to re-implementing Avro's (in a bounded timeframe especially), we should
review our requirements. This decision needs to be made for technical reasons and not grounded
in NIH.

After reviewing the Avro spec again, and having written a very similar file format for #674,
there is nothing I would change about the format:
 * Has a header to store file format version, compression information and any other arbitrary
data (including Avro's schema)
 * Is blocked based, with framing around the blocks for fast skipping, and with synchronization
points for recovering a corrupt file
 * Implements object reuse: iterating over a file requires a single record object, which is
re-filled with data in the file

In the interest of full disclosure, Avro is lacking one serialization feature I would like
to see (AVRO-679), but there is a fair chance it will be implemented in a future version,
and until then we can trivially implement it above Avro.

> Add bitmap secondary indexes
> ----------------------------
>                 Key: CASSANDRA-1472
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Stu Hood
>            Assignee: Stu Hood
>             Fix For: 0.7.1
>         Attachments: 0.7-1472-v5.tgz, 0.7-1472-v6.tgz, 0019-Rename-bugfixes-and-fileclose.txt,
1472-v3.tgz, 1472-v4.tgz, 1472-v5.tgz, anatomy.png, v4-bench-c32.txt
> Bitmap indexes are a very efficient structure for dealing with immutable data. We can
take advantage of the fact that SSTables are immutable by attaching them directly to SSTables
as a new component (supported by CASSANDRA-1471).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message