cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aleksey Yeschenko (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-9434) If a node loses schema_columns SSTables it could delete all secondary indexes from the schema
Date Tue, 18 Aug 2015 17:11:45 GMT


Aleksey Yeschenko commented on CASSANDRA-9434:

'Correctly' is not the word I would use. Expected, I guess.

I'll elaborate on what is going on. With CQL3 it became possible to give individual names
to the components of the key validator and the comparator. Those names (aliases) were originally
stored in {{system.schema_columnfamilies}} table, as JSON lists in columns {{key_aliases}}
and {{column_aliases}}. Only {{REGULAR}} columns were stored in {{system.schema_columns}}

Starting with 2.0 beta1 (CASSANDRA-5125) we keep all the 'aliases' in system.schema_columns
together with the regular columns - for the newly-created tables. That was done specifically
to allow secondary indexes on primary key columns.

CASSANDRA-6009 (in 2.0.1) added a start-up migration step so that previously created tables
(in 1.2 and before) too store their 'aliases' properly, in {{system.schema_columns}} ({{SystemKeyspace.copyAllAliasesToColumnsProper()}}).
With sstables for columns absent, it just treated the tables as requiring a migration in your
case, and so the new converted columns were created, and lacked any information about the
indexes - since that was never stored in the aliases, and couldn't be. And those changes then
propagated to the other nodes, and {{null}} s for {{index_info}}, {{index_options}}, and {{index_name}}
won, given the same timestamp.

This is a rare case (losing your columns sstables). This is properly fixed in 3.0, with Sam's
latest commit of CASSANDRA-6717, where the indexes are kept in a separate schema table.

2.1 and 2.2 might still be affected, and I think we can actually guard against it in 2.0.2
and up, but the logic for 2.0 will be somewhat involved. 2.1 and 2.2, OTOH, pretty straightforward.
Given that information, and 2.0 being EOL, do you object me only taking care of it in 2.1
and 2.2, if affected?

> If a node loses schema_columns SSTables it could delete all secondary indexes from the
> ---------------------------------------------------------------------------------------------
>                 Key: CASSANDRA-9434
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Richard Low
>            Assignee: Aleksey Yeschenko
>             Fix For: 2.0.x
> It is possible that a single bad node can delete all secondary indexes if it restarts
and cannot read its schema_columns SSTables. Here's a reproduction:
> * Create a 2 node cluster (we saw it on 2.0.11)
> * Create the schema:
> {code}
> create keyspace myks with replication = {'class':'SimpleStrategy', 'replication_factor':1};
> use myks;
> create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
> create index myindex on mytable(b);
> {code}
> NB index must be on clustering column to repro
> * Kill one node
> * Wipe its commitlog and system/schema_columns sstables.
> * Start it again
> * Run on this node
> select index_name from system.schema_columns where keyspace_name = 'myks' and columnfamily_name
= 'mytable' and column_name = 'b';
> and you'll see the index is null.
> * Run 'describe schema' on the other node. Sometimes it will not show the index, but
you might need to bounce for it to disappear.
> I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.

This message was sent by Atlassian JIRA

View raw message