cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uttam Phalnikar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7186) alter table add column not always propogating
Date Thu, 14 Apr 2016 19:01:25 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241725#comment-15241725
] 

Uttam Phalnikar commented on CASSANDRA-7186:
--------------------------------------------

We are experiencing similar issue intermittently. It usually happens when table has some data
(m+ records)

Steps to reproduce:
- Alter table add column
- nodetool describecluster to verify the nodes are in sync
- desc table from any node to verify column is added to the table
- select * from table limit 1 doesn't show the column
- insert into table (id, <new-col>)values('some-id', <col-value) throws following
error
InvalidRequest: code=2200 [Invalid query] message="Unknown identifier <new-col>"

So far system has fixed itself after couple of hours. Is there any way to find out where the
lag is or determine when the "internal sync" is done? 

Env Detail:
Spark	1.2.2.1
DSE	4.7.1
Cassandra	2.1.8.621
Search	4.10.3

> alter table add column not always propogating
> ---------------------------------------------
>
>                 Key: CASSANDRA-7186
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7186
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Martin Meyer
>            Assignee: Philip Thompson
>             Fix For: 2.0.12
>
>
> I've been many times in Cassandra 2.0.6 that adding columns to existing tables seems
to not fully propagate to our entire cluster. We add an extra column to various tables maybe
0-2 times a week, and so far many of these ALTERs have resulted in at least one node showing
the old table description a pretty long time (~30 mins) after the original ALTER command was
issued.
> We originally identified this issue when a connected clients would complain that a column
it issued a SELECT for wasn't a known column, at which point we have to ask each node to describe
the most recently altered table. One of them will not know about the newly added field. Issuing
the original ALTER statement on that node makes everything work correctly.
> We have seen this issue on multiple tables (we don't always alter the same one). It has
affected various nodes in the cluster (not always the same one is not getting the mutation
propagated). No new nodes have been added to the cluster recently. All nodes are homogenous
(hardware and software), running 2.0.6. We don't see any particular errors or exceptions on
the node that didn't get the schema update, only the later error from a Java client about
asking for an unknown column in a SELECT. We have to check each node manually to find the
offender. The tables he have seen this on are under fairly heavy read and write load, but
we haven't altered any tables that are not, so that might not be important.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message