Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 83629 invoked from network); 7 Feb 2011 20:09:14 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 7 Feb 2011 20:09:14 -0000 Received: (qmail 66481 invoked by uid 500); 7 Feb 2011 20:09:12 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 66430 invoked by uid 500); 7 Feb 2011 20:09:11 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 66422 invoked by uid 99); 7 Feb 2011 20:09:11 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Feb 2011 20:09:11 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [69.50.2.38] (HELO ht1.hostedexchange.local) (69.50.2.38) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Feb 2011 20:09:05 +0000 Received: from HT1.hostedexchange.local (172.16.69.39) by ht2.hostedexchange.local (172.16.69.40) with Microsoft SMTP Server (TLS) id 8.2.255.0; Mon, 7 Feb 2011 12:08:43 -0800 Received: from mbx2.hostedexchange.local ([172.16.69.30]) by HT1.hostedexchange.local ([172.16.69.39]) with mapi; Mon, 7 Feb 2011 12:08:43 -0800 From: Curt Allred To: "user@cassandra.apache.org" Date: Mon, 7 Feb 2011 12:07:03 -0800 Subject: RE: hadoop library conflict with cassandra (hector) 0.7 Thread-Topic: hadoop library conflict with cassandra (hector) 0.7 Thread-Index: AcvG/jqCGhujMst4QMmuiGqCdN6NagABDXxg Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F92FBB1CFC0C294E9BED0D3324BB6D75017B496594mbx2hostedexc_" MIME-Version: 1.0 --_000_F92FBB1CFC0C294E9BED0D3324BB6D75017B496594mbx2hostedexc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello, I'm trying to access a Cassandra 0.7 cluster in a hadoop map-reduce job (ha= doop 0.20.2) and seeing a thrift library conflict. Hadoop uses an older ve= rsion of thrift than hector 0.7, and this older version is getting picked u= p by my job, causing the following exception: FATAL org.apache.hadoop.mapred.TaskTracker: Error running child : java.lang.NoSuchMethodError: org.apache.thrift.protocol.TProtocol.writeBina= ry(Ljava/nio/ByteBuffer;)V The ByteBuffer overload of that method is not in the older thrift library. I have verified that the correct version of thrift (libthrift-0.5.0.jar) is= in my map-reduce job's jar. However its using the older version in the ha= doop library. Here are a couple of hadoop tickets on the issue: https://issues.apache.org/jira/browse/MAPREDUCE-1700 https://issues.apache.org/jira/browse/MAPREDUCE-1938 Anyone find a way around this? (besides modifying the hadoop cluster librar= y which is not an option for me). Thanks, -Curt --_000_F92FBB1CFC0C294E9BED0D3324BB6D75017B496594mbx2hostedexc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hello,

I'm trying to access a Cassandra 0.7 cluster in a= hadoop map-reduce job (hadoop 0.20.2) and seeing a thrift library conflict= .  Hadoop uses an older version of thrift than hector 0.7, and this ol= der version is getting picked up by my job, causing the following exception= :

 

FATAL org.apache.hadoop.mapred.TaskTracker: Error running child :<= /o:p>

java.lang.NoSuchMethodError: org.apache.thrif= t.protocol.TProtocol.writeBinary(Ljava/nio/ByteBuffer;)V

 

The ByteBuffer o= verload of that method is not in the older thrift library.

 

I have verifie= d that the correct version of thrift (libthrift-0.5.0.jar) is in my map-red= uce job's jar.  However its using the older version in the hadoop libr= ary. 

 

Here are a couple of hadoop tickets on the issue:<= /p>

https://issues.apache.org/jira/browse/MAPREDUCE-1700=

https://issues.apache.org/jira/browse/MAPREDUCE-1938

 

Anyone find a way around this? (besides modifying the hadoop cluster l= ibrary which is not an option for me). 

Thanks,

-Curt

 

= --_000_F92FBB1CFC0C294E9BED0D3324BB6D75017B496594mbx2hostedexc_--