Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7D4168ABA for ; Mon, 5 Sep 2011 14:42:05 +0000 (UTC) Received: (qmail 17242 invoked by uid 500); 5 Sep 2011 14:42:03 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 17185 invoked by uid 500); 5 Sep 2011 14:42:02 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 17177 invoked by uid 99); 5 Sep 2011 14:42:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Sep 2011 14:42:02 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL,TO_NO_BRKTS_PCNT X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: 209.85.214.44 is neither permitted nor denied by domain of oberman@civicscience.com) Received: from [209.85.214.44] (HELO mail-bw0-f44.google.com) (209.85.214.44) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Sep 2011 14:41:54 +0000 Received: by bkar4 with SMTP id r4so5510648bka.31 for ; Mon, 05 Sep 2011 07:41:32 -0700 (PDT) Received: by 10.204.130.135 with SMTP id t7mr2119837bks.256.1315233692728; Mon, 05 Sep 2011 07:41:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.152.129 with HTTP; Mon, 5 Sep 2011 07:41:12 -0700 (PDT) X-Originating-IP: [24.131.19.240] In-Reply-To: <585A8053-67F6-49F3-B7AC-2A24BDD3860A@gmail.com> References: <2A57A755-2367-4E02-A9A2-D65D9FE1579B@civicscience.com> <585A8053-67F6-49F3-B7AC-2A24BDD3860A@gmail.com> From: William Oberman Date: Mon, 5 Sep 2011 10:41:12 -0400 Message-ID: Subject: Re: cassandra 0.8.4 + pig (using cloudera rpms) To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=00151743f8040d3c9104ac32b5f9 --00151743f8040d3c9104ac32b5f9 Content-Type: text/plain; charset=ISO-8859-1 Yes, my cluster is working. I didn't realize it at the time, but the StorageService link I listed is already in 0.8.4, so yes the only file I had to patch was VersionedValue. Not sure what was going on with the pig jars, but after more configuration changes than I can count, I'm pretty sure removing pig.jar in favor of the cloudera pig jar was the magic bullet (for the ClassNotFound I was getting, in this case TException). One final note: in production I had to patch all of my cassandra servers (OLTP and analytics)* with the VersionedValue file for it to work (though, I did forget one setting, so now I'm still not 100% sure I had to patch all of them, but it's working now). will OLTP = vanilla cassandra analytics = cassandra + tasktracker. I'm not on brisk yet, so I've been rolling my own. On Mon, Sep 5, 2011 at 1:41 AM, Jeremy Hanna wrote: > Thanks William - so you were able to get everything running correctly, > right? > > FWIW, we're in the process of upgrading to 0.8.4 and found that all we > needed was that first link you mentioned - the VersionedValue modification. > It's running fine on our staging cluster and we're in the process of moving > to production. We're currently using pig from cdhu0. All we did was > replace the 0.8.4 jars after installing the debian packages for 0.8.4. > > Not sure if that helps anyone, but thought I would share what we've seen. > > Btw, this shouldn't be a problem once 0.8.5 comes out. > > On Sep 4, 2011, at 11:03 AM, William Oberman wrote: > > > I've had some troubles, so I thought I'd pass on my various bug fixes: > > > > -Cass 0.8.4 has troubles with pig/hadoop (you get NPE's when trying to > connect to cassandra in the pig logs). You need this patch: > > http://svn.apache.org/viewvc?revision=1158940&view=revision > > And maybe this: > > http://svn.apache.org/viewvc?revision=1155157&view=revision > > > > -I had installed from riptano rpms. I downloaded the src, applied the > patch, and did "ant jar". I then replaced the rpm installed cassandra jar > with this new one (ugly, but I wanted to continue to run from the package). > > > > -I think I was able to just replace the apache-cassandra-0.8.4.jar on > just my jobtracker + tasktracker nodes (I need to retest from scratch to be > sure, I've done a _lot_ of configuring and reconfiguring) > > > > -Then I started getting ClassNotFound exceptions during map/reduce tasks. > Still not sure why this fix works, but the problem seems to be cloudera pig > version 0.20.2+923.97-1 has two jars that match pig*.jar (which is what > cassandra contrib/pig/bin/pig_cassandra uses to setup the classpath). I had > to rename /usr/lib/pig/pig.jar for things to work (leaving > pig-0.8.1-cdh3u1-core.jar as the only match). > > > > My pig script is still running, but it's the first time it didn't > immediately crash. > > > > will > > > > > > --00151743f8040d3c9104ac32b5f9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Yes, my cluster is working. =A0

I didn't realize it = at the time, but the StorageService link I listed is already in 0.8.4, so y= es the only file I had to patch was VersionedValue. =A0Not sure what was go= ing on with the pig jars, but after more configuration changes than I can c= ount, I'm pretty sure removing pig.jar in favor of the cloudera pig jar= was the magic bullet (for the ClassNotFound I was getting, in this case TE= xception).

One final note: in production I had to patch all of my = cassandra servers (OLTP and analytics)* with the VersionedValue file for it= to work (though, I did forget one setting, so now I'm still not 100% s= ure I had to patch all of them, but it's working now).

will

OLTP =3D vanilla cassandr= a
analytics =3D cassandra + tasktracker. =A0I'm not on brisk = yet, so I've been rolling my own.


On Mon, Sep 5, 2011 at 1:41 AM, Jeremy Hanna <jeremy.hanna1234@gmail.com> wrote:
Thanks William - so you were able to get everything running correctly, righ= t?

FWIW, we're in the process of upgrading to 0.8.4 and found that all we = needed was that first link you mentioned - the VersionedValue modification.= =A0It's running fine on our staging cluster and we're in the proce= ss of moving to production. =A0We're currently using pig from cdhu0. = =A0All we did was replace the 0.8.4 jars after installing the debian packag= es for 0.8.4.

Not sure if that helps anyone, but thought I would share what we've see= n.

Btw, this shouldn't be a problem once 0.8.5 comes out.

On Sep 4, 2011, at 11:03 AM, William Oberman wrote:

> I've had some troubles, so I thought I'd pass on my various bu= g fixes:
>
> -Cass 0.8.4 has troubles with pig/hadoop (you get NPE's when tryin= g to connect to cassandra in the pig logs). =A0You need this patch:
> http://svn.apache.org/viewvc?revision=3D1158940= &view=3Drevision
> And maybe this:
> http://svn.apache.org/viewvc?revision=3D1155157= &view=3Drevision
>
> -I had installed from riptano rpms. =A0I downloaded the src, applied t= he patch, and did "ant jar". =A0I then replaced the rpm installed= cassandra jar with this new one (ugly, but I wanted to continue to run fro= m the package).
>
> -I think I was able to just replace the apache-cassandra-0.8.4.jar on = just my jobtracker + tasktracker nodes (I need to retest from scratch to be= sure, I've done a _lot_ of configuring and reconfiguring)
>
> -Then I started getting ClassNotFound exceptions during map/reduce tas= ks. =A0Still not sure why this fix works, but the problem seems to be cloud= era pig version 0.20.2+923.97-1 has two jars that match pig*.jar (which is = what cassandra contrib/pig/bin/pig_cassandra uses to setup the classpath). = =A0I had to rename /usr/lib/pig/pig.jar for things to work (leaving pig-0.8= .1-cdh3u1-core.jar as the only match).
>
> My pig script is still running, but it's the first time it didn= 9;t immediately crash.
>
> will
>
>



--00151743f8040d3c9104ac32b5f9--