Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 34422 invoked from network); 30 Sep 2010 23:38:03 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 30 Sep 2010 23:38:03 -0000 Received: (qmail 78961 invoked by uid 500); 30 Sep 2010 23:38:01 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 78922 invoked by uid 500); 30 Sep 2010 23:38:00 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 78914 invoked by uid 99); 30 Sep 2010 23:38:00 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Sep 2010 23:38:00 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a46.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Sep 2010 23:37:55 +0000 Received: from homiemail-a46.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a46.g.dreamhost.com (Postfix) with ESMTP id CA44F3E406C for ; Thu, 30 Sep 2010 16:37:33 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=to:from :subject:date:message-id:content-type:mime-version:in-reply-to; q=dns; s=thelastpickle.com; b=DdTOcRQq2CYyqLryOo+l4hSc/W2aZHBs4 ncw9tbnxm0XB0s0OUO4G/fcku1USfKyFb1ot+Jqd//GQFiEO1ehqmxktXt/U8cdL buMUtPLnFi7jxLRlLRzD27wROj2X9HZ6QJK3m3btgHc+0vw+pqO3mWjzRUXf1VNW +PRzTSfpA0= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=to :from:subject:date:message-id:content-type:mime-version: in-reply-to; s=thelastpickle.com; bh=iihAlf8HU5qtb+1L+b3aMaUKsZc =; b=1gT8WzaiNrhKVgN7dmmx7Vvno22NkjSDfUF0nnoQj9cfHpIHC4EXYj9YqXw q+0lb19S/neAwdMhBHyCCZSqqPNoYB+tE0r6IEvDYcu0ZNu5viv/no7mPCHkQN/6 0vel19xwn31i4r97tBT6t8ffOq2uwdMrdVCm/g1S/pv6N1bw= Received: from localhost (webms.mac.com [17.148.16.116]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a46.g.dreamhost.com (Postfix) with ESMTPSA id B33D43E4065 for ; Thu, 30 Sep 2010 16:37:33 -0700 (PDT) To: user@cassandra.apache.org From: Aaron Morton Subject: Re: Schema Migration Failed Date: Thu, 30 Sep 2010 23:37:32 GMT X-Mailer: MobileMe Mail (1C3203) Message-id: <6ec1b058-7be4-69e5-ccb3-8671a0208a82@me.com> Content-Type: multipart/alternative; boundary=Apple-Webmail-42--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d MIME-Version: 1.0 In-Reply-To: --Apple-Webmail-42--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; format=flowed OK, turns out the sky was not falling :)=0A=0Adriftx helped me on IRC, we = added the node tokens to the yaml file then deleted the entire system dire= ctory, restarted and it worked.=A0=0A=0AGuess I need to go back and read=A0= http://www.riptano.com/blog/live-schema-updates-cassandra-07=A0to understa= nd schema propagation.=A0=0A=0AThanks=0Aaaron=0A=0AOn 01 Oct, 2010,at 11:3= 7 AM, Aaron Morton wrote:=0A=0AUpgrading from 0.= 7 nightly for 31/08 to beta 2.=A0=0A=0AThis one seems serious, schema migr= ation is failing. I'm dead in the water here, so any quick ideas appreciat= ed.=A0=0A=0AHave done an import schema on one node, and also tried system_= add_keyspace on the same node. It's showing the Keyspace1 and my keyspace.= But am getting these errors on the other nodes...=0A=0AERROR [READ_STAGE:= 19] 2010-10-01 11:31:57,872 AbstractCassandraDaemon.java (line 88) Fatal e= xception in thread Thread[READ_STAGE:19,5,main]=0Ajava.lang.ClassCastExcep= tion: org.apache.avro.util.Utf8 cannot be cast to java.lang.Double=0A=A0=A0= =A0 =A0 =A0at org.apache.cassandra.config.avro.CfDef.put(CfDef.java:60)=0A= =A0=A0 =A0 =A0 =A0at org.apache.avro.generic.GenericDatumReader.setField(G= enericDatumReader.java:152)=0A=A0=A0 =A0 =A0 =A0at org.apache.avro.generic= GenericDatumReader.readRecord(GenericDatumReader.java:142)=0A=A0=A0 =A0 =A0= =A0at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.= java:114)=0A=A0=A0 =A0 =A0 =A0at org.apache.avro.generic.GenericDatumReade= r.readArray(GenericDatumReader.java:192)=0A=A0=A0 =A0 =A0 =A0at org.apache= avro.generic.GenericDatumReader.read(GenericDatumReader.java:116)=0A=A0=A0= =A0 =A0 =A0at org.apache.avro.generic.GenericDatumReader.readRecord(Gener= icDatumReader.java:142)=0A=A0=A0 =A0 =A0 =A0at org.apache.avro.generic.Gen= ericDatumReader.read(GenericDatumReader.java:114)=0A=A0=A0 =A0 =A0 =A0at o= rg.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.ja= va:142)=0A=A0=A0 =A0 =A0 =A0at orgapache.avro.generic.GenericDatumReader.r= ead(GenericDatumReader.java:114)=0A=A0=A0 =A0 =A0 =A0at org.apache.avro.ge= neric.GenericDatumReader.read(GenericDatumReader.java:118)=0A=A0=A0 =A0 =A0= =A0at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumR= eader.java:142)=0A=A0=A0 =A0 =A0 =A0at org.apache.avro.generic.GenericDatu= mReader.read(GenericDatumReader.java:114)=0A=A0=A0 =A0 =A0 =A0at org.apach= e.avro.genericGenericDatumReader.read(GenericDatumReader.java:105)=0A=A0=A0= =A0 =A0 =A0at org.apache.cassandra.io.SerDeUtils.deserializeWithSchema(Se= rDeUtils.java:98)=0A=A0=A0 =A0 =A0 =A0at org.apache.cassandra.db.migration= Migration.deserialize(Migration.java:262)=0A=A0=A0 =A0 =A0 =A0at org.apac= he.cassandra.db.DefinitionsUpdateResponseVerbHandler.doVerb(DefinitionsUpd= ateResponseVerbHandler.java:57)=0A=A0=A0 =A0 =A0 =A0at org.apache.cassandr= a.net.MessageDeliveryTask.run(MessageDeliveryTask.java:50)=0A=A0=A0 =A0 =A0= =A0at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolEx= ecutor.java:886)=0A=A0=A0 =A0 =A0 =A0at java.util.concurrent.ThreadPoolExe= cutor$Worker.run(ThreadPoolExecutor.java:908)=0A=A0=A0 =A0 =A0 =A0at java.= lang.Thread.run(Thread.java:619)=0A=0AAnd they are not showing the keyspac= e in JMX. So things are way out of wack.=0A=0Ais it correct to see avro in= the call stack ?=A0=0A=0AAaron=0A=0A --Apple-Webmail-42--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d Content-Type: multipart/related; type="text/html"; boundary=Apple-Webmail-86--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d --Apple-Webmail-86--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1;
OK, turns out the sky was not falling :)

dri= ftx helped me on IRC, we added the node tokens to the yaml file then delet= ed the entire system directory, restarted and it worked. 
<= br>
Guess I need to go back and read http://www.riptano.com= /blog/live-schema-updates-cassandra-07 to understand schema propa= gation. 

Thanks
aaron

On 01= Oct, 2010,at 11:37 AM, Aaron Morton <aaron@thelastpickle.com> wrote= :

Upgrading from 0.7= nightly for 31/08 to beta 2. 

This one seem= s serious, schema migration is failing. I'm dead in the water here, so any= quick ideas appreciated. 

Have done an impo= rt schema on one node, and also tried system_add_keyspace on the same node= It's showing the Keyspace1 and my keyspace. But am getting these errors = on the other nodes...

ERROR [READ_STAGE:19] = 2010-10-01 11:31:57,872 AbstractCassandraDaemon.java (line 88) Fatal excep= tion in thread Thread[READ_STAGE:19,5,main]
java.lang.ClassCastE= xception: org.apache.avro.util.Utf8 cannot be cast to java.lang.Double
        at org.apache.cassandra.config.avr= o.CfDef.put(CfDef.java:60)
        at o= rg.apache.avro.generic.GenericDatumReader.setField(GenericDatumReader.java= :152)
        at org.apache.avro.generi= c.GenericDatumReader.readRecord(GenericDatumReader.java:142)
&nb= sp;       at org.apache.avro.generic.GenericDatumReade= r.read(GenericDatumReader.java:114)
       &= nbsp;at org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumR= eader.java:192)
        at org.apache.a= vro.generic.GenericDatumReader.read(GenericDatumReader.java:116)
        at org.apache.avro.generic.GenericDatumR= eader.readRecord(GenericDatumReader.java:142)
    = ;    at org.apache.avro.generic.GenericDatumReader.read(GenericD= atumReader.java:114)
        at org.apa= che.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:142= )
        at orgapache.avro.generic.Gen= ericDatumReader.read(GenericDatumReader.java:114)
   &= nbsp;    at org.apache.avro.generic.GenericDatumReader.read(Gene= ricDatumReader.java:118)
        at org= apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java= :142)
        at org.apache.avro.generi= c.GenericDatumReader.read(GenericDatumReader.java:114)
 &nb= sp;      at org.apache.avro.genericGenericDatumReader.read(= GenericDatumReader.java:105)
        at= org.apache.cassandra.io.SerDeUtils.deserializeWithSchema(SerDeUtils.java:= 98)
        at org.apache.cassandra.db.= migration.Migration.deserialize(Migration.java:262)
  =      at org.apache.cassandra.db.DefinitionsUpdateResponseV= erbHandler.doVerb(DefinitionsUpdateResponseVerbHandler.java:57)
=         at org.apache.cassandra.net.MessageDelive= ryTask.run(MessageDeliveryTask.java:50)
     &nbs= p;  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadP= oolExecutor.java:886)
        at java.u= til.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)<= /div>
        at java.lang.Thread.run(Thread.= java:619)

And they are not showing the keys= pace in JMX. So things are way out of wack.

is it= correct to see avro in the call stack ? 

Aa= ron

--Apple-Webmail-86--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d-- --Apple-Webmail-42--77e866c7-6456-e3fd-bbd6-c0a6bbbfd72d--