Return-Path: X-Original-To: apmail-crunch-user-archive@www.apache.org Delivered-To: apmail-crunch-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 127E410E59 for ; Tue, 25 Feb 2014 19:55:46 +0000 (UTC) Received: (qmail 62793 invoked by uid 500); 25 Feb 2014 19:55:44 -0000 Delivered-To: apmail-crunch-user-archive@crunch.apache.org Received: (qmail 62464 invoked by uid 500); 25 Feb 2014 19:55:43 -0000 Mailing-List: contact user-help@crunch.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@crunch.apache.org Delivered-To: mailing list user@crunch.apache.org Received: (qmail 62348 invoked by uid 99); 25 Feb 2014 19:55:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 19:55:42 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of josh.wills@gmail.com designates 209.85.215.41 as permitted sender) Received: from [209.85.215.41] (HELO mail-la0-f41.google.com) (209.85.215.41) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 19:55:36 +0000 Received: by mail-la0-f41.google.com with SMTP id gl10so4550960lab.14 for ; Tue, 25 Feb 2014 11:55:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=u0NFQFnGhG9pV7NzIy4S46BdGHigfdPTjK81GfpNAho=; b=zHfHJd9KtpyI1KL83d3dhEE7X7Tq+lbvEd1/smaix5otMORQkMKyppiHr6U1sCoj34 imA3K0Mptenyvw+I4+2Z9SFzLMPUZp36nAFQT1DQ7BwAMzGH2cnMkpF42CnUtyxlLy29 +fJuOEf1BUUyEnjxPVFMzojKUtvI5nCOrCEFD4KVY0Wi1beAqpLcAh5Ty/+LMOB/q/B9 cSS7Edmozgz7QgdYyXcdy6WMFm1yM00/f9cw9wO0gAZQrtqMvFfMw0S6tBKdeVvj/Bi6 BbUN98S7xsqd4yfwFFt3Henn4Jl0o+j75mscwS7+HK0NFx3rayaSZ/uwEWf36gH7flhH IhUw== X-Received: by 10.112.170.234 with SMTP id ap10mr804444lbc.23.1393358116156; Tue, 25 Feb 2014 11:55:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.33.171 with HTTP; Tue, 25 Feb 2014 11:54:56 -0800 (PST) In-Reply-To: References: From: Josh Wills Date: Tue, 25 Feb 2014 11:54:56 -0800 Message-ID: Subject: Re: Yarnchild error : crunch-0.7.0 To: user@crunch.apache.org Content-Type: multipart/alternative; boundary=001a11c373ec8efbdd04f3407798 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c373ec8efbdd04f3407798 Content-Type: text/plain; charset=ISO-8859-1 Yeah, try it again w/ -Dcrunch.platform=2 instead of -Dhadoop.profile=2.0 J On Tue, Feb 25, 2014 at 11:47 AM, Bill Sparks wrote: > Well I did the following and also changed the pom.xml to reference the > correct hadoop version. > > $ mvn clean install -Dhadoop.profile=2.0 -DskipTests > > 2.0.6-alpha > > -- > Jonathan (Bill) Sparks > Software Architecture > Cray Inc. > > From: Josh Wills > Reply-To: "user@crunch.apache.org" > Date: Tuesday, February 25, 2014 1:43 PM > To: "user@crunch.apache.org" > Subject: Re: Yarnchild error : crunch-0.7.0 > > Hrm-- that's usually related to the API changes between hadoop1 and > hadoop2. How did you build crunch, exactly? Did you use > -Dcrunch.platform=2? > > J > > > On Tue, Feb 25, 2014 at 11:37 AM, Bill Sparks wrote: > >> Can anyone shed some light on why I would be getting the following >> error when submitting a simple crunch wordcount example. Other Hadoop MR >> applications work, just it seems that Crunch is confused about some class >> definitions. >> >> I'm running hadoop-2.0.6-alpha and have build crunch to match. >> >> Hadoop 2.0.6-alpha >> Subversion Unknown -r ca4c88898f95aaab3fd85b5e9c194ffd647c2109 >> Compiled by jenkins on 2013-10-30T07:19Z >> From source with checksum 95e88b2a9589fa69d6d5c1dbd48d4e >> >> >> 2014-02-25 13:23:00,049 INFO [main] org.apache.hadoop.mapred.MapTask: >> bufstart = 0; bufvoid = 268435456 >> 2014-02-25 13:23:00,049 INFO [main] org.apache.hadoop.mapred.MapTask: >> kvstart = 67108860; length = 16777216 >> 2014-02-25 13:23:00,070 FATAL [main] org.apache.hadoop.mapred.YarnChild: >> Error running child : java.lang.IncompatibleClassChangeError: >> org/apache/hadoop/mapreduce/JobContext.getConfiguration()Lorg/apache/hadoop/conf/Configuration; >> at >> org.apache.crunch.impl.mr.run.CrunchTaskContext.(CrunchTaskContext.java:42) >> at org.apache.crunch.impl.mr.run.CrunchMapper.setup(CrunchMapper.java:37) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:757) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) >> at java.security.AccessController.doPrivileged(AccessController.java:366) >> at javax.security.auth.Subject.doAs(Subject.java:572) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153) >> >> > > > -- > Director of Data Science > Cloudera > Twitter: @josh_wills > --001a11c373ec8efbdd04f3407798 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Yeah, try it again w/ -Dcrunch.platform=3D2 instead of -Dh= adoop.profile=3D2.0

J


On Tue, Feb 25, 2014 at 11:47 AM, Bi= ll Sparks <jsparks@cray.com> wrote:
Well I did the following and also changed the pom.xml to reference the= correct hadoop version.

$ mvn clean install -Dhadoop.profile=3D2.0 –DskipTests

<hadoop.version>2.0.6-alpha</hadoop.version>

-- 
Jonathan (Bill) Sparks
Software Architecture
Cray Inc.
From: Josh Wills <jwills@cloudera.com>
Reply-To: "user@crunch.apache.org" &= lt;user@crunch.= apache.org>
Date: Tuesday, February 25, 2014 1:= 43 PM
To: "user@crunch.apache.org" <user@crunch.apache= .org>
Subject: Re: Yarnchild error : crun= ch-0.7.0

Hrm-- that's usually related to the API changes betwee= n hadoop1 and hadoop2. How did you build crunch, exactly? Did you use -Dcru= nch.platform=3D2?

J


On Tue, Feb 25, 2014 at 11:37 AM, Bill Sparks <jsparks@cray.com<= /a>> wrote:
Can anyone shed some light on why I would be getting the following err= or when submitting a simple crunch wordcount example. Other Hadoop MR appli= cations work, just it seems that Crunch is confused about some class defini= tions.

I'm running hadoop-2.0.6-alpha and have build crunch to match.

Hadoop 2.0.6-alpha
Subversion Unknown -r ca4c88898f95aaab3fd85b5e9c194ffd647c2109
Compiled by jenkins on 2013-10-30T07:19Z
From source with checksum 95e88b2a9589fa69d6d5c1dbd48d4e


2014-02-25 13:23:00,049 INFO [main] org.apache.hadoop.mapred.MapTask: = bufstart =3D 0; bufvoid =3D 268435456
2014-02-25 13:23:00,049 INFO [main] org.apache.hadoop.mapred.MapTask: = kvstart =3D 67108860; length =3D 16777216
2014-02-25 13:23:00,070 FATAL [main] org.apache.hadoop.mapred.YarnChil= d: Error running child : java.lang.IncompatibleClassChangeError: org/apache= /hadoop/mapreduce/JobContext.getConfiguration()Lorg/apache/hadoop/conf/Conf= iguration;
at org.apache.crunch.impl.= mr.run.CrunchTaskContext.<init>(CrunchTaskContext.java:42)
at org.apache.crunch.impl.= mr.run.CrunchMapper.setup(CrunchMapper.java:37)
at org.apache.hadoop.mapre= duce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapre= d.MapTask.runNewMapper(MapTask.java:757)
at org.apache.hadoop.mapre= d.MapTask.run(MapTask.java:339)
at org.apache.hadoop.mapre= d.YarnChild$2.run(YarnChild.java:158)
at java.security.AccessCon= troller.doPrivileged(AccessController.java:366)
at javax.security.auth.Sub= ject.doAs(Subject.java:572)
at org.apache.hadoop.secur= ity.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at org.apache.hadoop.mapre= d.YarnChild.main(YarnChild.java:153)




--
Director of Data Science

--001a11c373ec8efbdd04f3407798--