incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Frank <...@airlust.com>
Subject Re: Schema consistently not propagating to a node.
Date Thu, 13 Sep 2012 18:53:32 GMT
Hi Sergey,
   That was exactly it, thank you!

-Ben

On Thu, Sep 13, 2012 at 12:07 AM, Sergey Tryuber <stryuber@gmail.com> wrote:

> Hi Ben,
>
> We've seen similar behavior once a time. The cause was unsynchronized time
> on the nodes. So I can recommend to check that system time is equal over
> the nodes (especially on that one "bad" server).
>
>
> On 13 September 2012 05:15, Ben Frank <ben@airlust.com> wrote:
>
>> Hey all,
>>    I'm setting up a new ring on 11 machines with cassandra 1.1.5. All
>> seems to install fine and startup ok (from tarball) but I'm having an issue
>> when updating the schema. There's one node that just doesn't want to
>> receive the schema change. I've tried blowing away my /var/lib/cassandra/*
>> directories and trying it a few times with the same results. I also tried
>> the instructions here:
>> http://wiki.apache.org/cassandra/FAQ#schema_disagreement
>>
>> If I update the schema with cassandra-cli I get the following error:
>>
>> Waiting for schema agreement...
>> The schema has not settled in 10 seconds; further migrations are
>> ill-advised until it does.
>> Versions are 3b172ce8-d8e8-362f-b955-79fe6b8a35e4:[12.19.103.103,
>> 12.19.110.103, 12.19.102.137, 12.19.103.111, 12.19.102.131, 12.19.102.129,
>> 12.19.110.105, 172.19.102.135, 12.19.103.105, 12.19.102.133],
>> b83fe28a-2851-34cb-bd8d-7d2621c1d872:[12.19.110.109]
>>
>> then if I do a 'describe cluster' I get:
>>
>> Cluster Information:
>>    Snitch: org.apache.cassandra.locator.SimpleSnitch
>>    Partitioner: org.apache.cassandra.dht.RandomPartitioner
>>    Schema versions:
>> 3b172ce8-d8e8-362f-b955-79fe6b8a35e4: [12.19.103.103, 12.19.110.103,
>> 12.19.102.137, 12.19.103.111, 12.19.102.131, 12.19.102.129, 12.19.110.105,
>> 12.19.102.135, 12.19.103.105, 12.19.102.133]
>>
>> a0f80d21-ad98-31df-b6dc-871061b1bcf9: [12.19.110.109]
>>
>> if I keep running describe cluster, both the schema UUID's(?) above keep
>> changing every few seconds. Looking at the logs shows the nodes
>> are continually doing a flush and compact.
>>
>> if I start over by blowing away /var/lib/cassandra, and then try the
>> schema update on the machine which is having trouble receiving
>> it everything works ok and all nodes are in sync.
>>
>> Anyone know what's going on here, or what I should look at to
>> troubleshoot this?
>>
>> -Ben
>>
>
>

Mime
View raw message