cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sam Tunnicliffe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-7268) Secondary Index can miss data without an error
Date Mon, 16 Jun 2014 19:53:03 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sam Tunnicliffe updated CASSANDRA-7268:
---------------------------------------

    Attachment: 7268-2.0.txt
                7268-2.1.txt
                7268-1.2.txt

The problem here is that we have two expiring columns which differ only in localExpiryTime
values and which are not considered equal (correctly). When an overwrite like this occurs
it leads to the insertion of a tombstone which having the same timestamp as both the new and
old columns, effectively hides the indexed value.

This isn't a problem in 2.1 because the reconcilliation logic there is reversed. So in this
case, the old value is retained and the index update never happens. While this masks the problem
described here, it causes its own problems - see CASSANDRA-7403 

I've attached patches to add some additional checks before inserting the index tombstone (unfortunately
there are a number of incompatibilities between 1.2/2.0/2.1 in this area of the code, so there's
a patch per-branch). I've also added a unit test for the overwrite behaviour and run this
through some manual testing of compactions & everything seems to be working ok. 

> Secondary Index can miss data without an error
> ----------------------------------------------
>
>                 Key: CASSANDRA-7268
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7268
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Jeremiah Jordan
>            Assignee: Sam Tunnicliffe
>         Attachments: 7268-1.2.txt, 7268-2.0.txt, 7268-2.1.txt
>
>
> Seeing issues with secondary indexes after upgrading from 1.1->1.2.  Using the same
thrift code from 1.1, every once in a while a row is inserted that does not show up in the
secondary index on a text column.
> Using sstable2json we can see the row in the regular sstables on every node, but not
in the secondary index sstables (even after flushing/taking a snapshot).
> If we move the snapshot to a test node and rebuild the secondary index, it gets populated
correctly and returns the data.
> Sanitized create statement:
> {noformat}
> create column family test2i 
> with column_type = 'Standard' 
> and comparator = 'UTF8Type' 
> and default_validation_class = 'UTF8Type' 
> and key_validation_class = 'UTF8Type' 
> and read_repair_chance = 1.0 
> and dclocal_read_repair_chance = 0.0 
> and populate_io_cache_on_flush = false 
> and gc_grace = 0 
> and min_compaction_threshold = 4 
> and max_compaction_threshold = 32 
> and replicate_on_write = false 
> and compaction_strategy = 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'

> and caching = 'KEYS_ONLY' 
> and column_metadata = [ 
> {column_name : 'second', 
> validation_class : UTF8Type, 
> index_name : 'test2i_second_idx', 
> index_type : 0}, 
> {column_name : 'A', 
> validation_class : UTF8Type}, 
> {column_name : 'B', 
> validation_class : UTF8Type}, 
> {column_name : 'C', 
> validation_class : UTF8Type}, 
> {column_name : 'D', 
> validation_class : UTF8Type}, 
> {column_name : 'E', 
> validation_class : UTF8Type}, 
> {column_name : 'F', 
> validation_class : UTF8Type}, 
> {column_name : 'G', 
> validation_class : UTF8Type}, 
> {column_name : 'H', 
> validation_class : UTF8Type}, 
> {column_name : 'I', 
> validation_class : UTF8Type}, 
> {column_name : 'J', 
> validation_class : UTF8Type}, 
> {column_name : 'K', 
> validation_class : UTF8Type}, 
> {column_name : 'L', 
> validation_class : UTF8Type}, 
> {column_name : 'M', 
> validation_class : UTF8Type}] 
> and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'};
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message