incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Tryuber <stryu...@gmail.com>
Subject Re: Schema consistently not propagating to a node.
Date Thu, 13 Sep 2012 07:07:40 GMT
Hi Ben,

We've seen similar behavior once a time. The cause was unsynchronized time
on the nodes. So I can recommend to check that system time is equal over
the nodes (especially on that one "bad" server).


On 13 September 2012 05:15, Ben Frank <ben@airlust.com> wrote:

> Hey all,
>    I'm setting up a new ring on 11 machines with cassandra 1.1.5. All
> seems to install fine and startup ok (from tarball) but I'm having an issue
> when updating the schema. There's one node that just doesn't want to
> receive the schema change. I've tried blowing away my /var/lib/cassandra/*
> directories and trying it a few times with the same results. I also tried
> the instructions here:
> http://wiki.apache.org/cassandra/FAQ#schema_disagreement
>
> If I update the schema with cassandra-cli I get the following error:
>
> Waiting for schema agreement...
> The schema has not settled in 10 seconds; further migrations are
> ill-advised until it does.
> Versions are 3b172ce8-d8e8-362f-b955-79fe6b8a35e4:[12.19.103.103,
> 12.19.110.103, 12.19.102.137, 12.19.103.111, 12.19.102.131, 12.19.102.129,
> 12.19.110.105, 172.19.102.135, 12.19.103.105, 12.19.102.133],
> b83fe28a-2851-34cb-bd8d-7d2621c1d872:[12.19.110.109]
>
> then if I do a 'describe cluster' I get:
>
> Cluster Information:
>    Snitch: org.apache.cassandra.locator.SimpleSnitch
>    Partitioner: org.apache.cassandra.dht.RandomPartitioner
>    Schema versions:
> 3b172ce8-d8e8-362f-b955-79fe6b8a35e4: [12.19.103.103, 12.19.110.103,
> 12.19.102.137, 12.19.103.111, 12.19.102.131, 12.19.102.129, 12.19.110.105,
> 12.19.102.135, 12.19.103.105, 12.19.102.133]
>
> a0f80d21-ad98-31df-b6dc-871061b1bcf9: [12.19.110.109]
>
> if I keep running describe cluster, both the schema UUID's(?) above keep
> changing every few seconds. Looking at the logs shows the nodes
> are continually doing a flush and compact.
>
> if I start over by blowing away /var/lib/cassandra, and then try the
> schema update on the machine which is having trouble receiving
> it everything works ok and all nodes are in sync.
>
> Anyone know what's going on here, or what I should look at to troubleshoot
> this?
>
> -Ben
>

Mime
View raw message