cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Dusbabek (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (CASSANDRA-1472) Add bitmap secondary indexes
Date Sat, 12 Feb 2011 13:51:57 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983248#comment-12983248
] 

Gary Dusbabek edited comment on CASSANDRA-1472 at 2/12/11 1:50 PM:
-------------------------------------------------------------------

bq. would it be reasonable to commit this patch with Avro's file format, and then to discuss
replacing the file format in a separate issue
Not a good idea, imo. 

bq. writing our own file format is an optimization
I don't think it is an optimization at all.  It's a way of protecting us from ourselves. 
The only way to do that and still use avro is to create our own readers according to the spec,
no?

bq.  This decision needs to be made for technical reasons and not grounded in NIH.
The reasons are technical.  The fact is that the way we use avro is constantly breaking on
us (I'm referring to migrations).  Some of this is probably self-inflicted.  We don't have
much experience yet with what happens when records change slightly or radically over time
and how avro copes with that.  The experience we do have leads me to believe that unless we
are extremely cautious, we'll end up in a hole.  See CASSANDRA-2001 for an example.  Two seemly
innocuous changes have both managed to break migration deserialization.

      was (Author: gdusbabek):
    bq. would it be reasonable to commit this patch with Avro's file format, and then to discuss
replacing the file format in a separate issue
Not a good idea, imo. 

bq. writing our own file format is an optimization
I don't think it is an optimization at all.  It's a way of protecting us from ourselves. 
The only way to do that and still use avro is to create our own readers according to the spec,
no?

bq.  This decision needs to be made for technical reasons and not grounded in NIH.
Poppycock.  The reasons are technical.  The fact is that the way we use avro is constantly
breaking on us (I'm referring to migrations).  Some of this is probably self-inflicted, but
either way--we should avoid chopping down the forest until we can operate the chainsaw without
cutting off our fingers.  We don't have much experience yet with what happens when records
change slightly or radically over time and how avro copes with that.  The experience we do
have leads me to believe that unless we are extremely cautious, we'll end up in a hole.  See
CASSANDRA-2001 for an example.  Two seemly innocuous changes have both managed to break migration
deserialization.
  
> Add bitmap secondary indexes
> ----------------------------
>
>                 Key: CASSANDRA-1472
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1472
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Stu Hood
>            Assignee: Stu Hood
>             Fix For: 0.7.2
>
>         Attachments: 0.7-1472-v5.tgz, 0.7-1472-v6.tgz, 0001-CASSANDRA-1472-rebased-to-0.7-branch.txt,
0019-Rename-bugfixes-and-fileclose.txt, 1472-v3.tgz, 1472-v4.tgz, 1472-v5.tgz, anatomy.png,
v4-bench-c32.txt
>
>
> Bitmap indexes are a very efficient structure for dealing with immutable data. We can
take advantage of the fact that SSTables are immutable by attaching them directly to SSTables
as a new component (supported by CASSANDRA-1471).

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message