pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohini Palaniswamy <rohini.adi...@gmail.com>
Subject Re: Pig 0.14. and Hadoop 2.0.0
Date Fri, 06 Feb 2015 19:03:34 GMT
java.lang.NoSuchMethodError: org.apache.hadoop.mapred.
jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/
jobcontrol/Job;)Ljava/lang/String;

   This error usually happens when you use Pig compiled with
-Dhadoopversion=23 with Hadoop 1.x mapreduce or -Dhadoopversion=20 with
Hadoop 2.x mapreduce cluster.

Regards,
Rohini


On Thu, Feb 5, 2015 at 7:51 AM, Carlos Balduz Bernal <cbalduz@gmail.com>
wrote:

> Hi,
>
> Our cluster uses Hortonworks HDP 2.2 with Hadoop 2.6.0 and Pig 0.14 and we
> haven't faced any problem yet.
>
> Regards,
>
> Carlos Balduz.
>
> On 5 February 2015 at 16:37, Quang-Nhat HOANG-XUAN <hxquangnhat@gmail.com>
> wrote:
>
> > Hi,
> > I faced this problem before and still can not fix it. My cluster is
> hadoop
> > 2.0.0-cdh4.4.0, it can work with Pig 13.0 but can not with Pig 14.0.
> > CDH distro is very complicated because it packages both mapreduce1 and
> > mapreduce2.
> > The only solution I got is waiting for Cloudera to upgrade their Pig or
> you
> > have to use a Hadoop distro from Apache.
> >
> > Quang-Nhat
> >
> > On Thu, Feb 5, 2015 at 4:20 PM, Michael Bohn <michael.bohn@freiheit.com>
> > wrote:
> >
> > > We updated (successful) our Pig to 0.12 and it worked.
> > >
> > > But it shouldn't matter where Hadoop comes from? I guess this problem
> > > exists with
> > > every combination of Hadoop 2.0 and Pig 0.14?
> > >
> > > Regards,
> > > Michael
> > >
> > > On 05.02.2015, at 16:15, Russell Jurney <russell.jurney@gmail.com>
> > wrote:
> > >
> > > CDH pig is the only Pig that works with CDH. In fact, it has to be the
> > > exact pig for that release of CDH. I tried very, very hard to use pig
> 12
> > > with CDH 4.x, and pig 13/14 with CDH 5.x, and I believe it is not
> > possible.
> > >
> > > Bug Cloudera to upgrade their Pig. Sorry :(
> > >
> > > On Thursday, February 5, 2015, Michael Bohn <michael.bohn@freiheit.com
> >
> > > wrote:
> > >
> > > Hi,
> > >
> > > we are trying to use the latest Pig-Version 0.14 with our Hadoop 2.0.0
> > > installation.
> > > We are using: 2.0.0-cdh4.7.0.jar
> > >
> > > This version seems not to be compatible with Pig 0.14.
> > >
> > > Even this minimal script:
> > >
> > > D = load 'test.txt';
> > > dump D;
> > >
> > > and test.txt an empty file, fails with the error:
> > >
> > > 2015-02-05 12:57:51,133 [main] WARN
> org.apache.hadoop.conf.Configuration
> > > - dfs.safemode.extension is deprecated. Instead, use
> > > dfs.namenode.safemode.extension
> > > Pig Stack Trace
> > > ---------------
> > > ERROR 2998: Unhandled internal error.
> > >
> > >
> >
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
> > >
> > > java.lang.NoSuchMethodError:
> > >
> > >
> >
> org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
> > >        at
> > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:325)
> > >        at
> > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:196)
> > >        at
> > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:280)
> > >        at org.apache.pig.PigServer.launchPlan(PigServer.java:1390)
> > >        at
> > >
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375)
> > >        at org.apache.pig.PigServer.storeEx(PigServer.java:1034)
> > >        at org.apache.pig.PigServer.store(PigServer.java:997)
> > >        at org.apache.pig.PigServer.openIterator(PigServer.java:910)
> > >        at
> > >
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:746)
> > >        at
> > >
> > >
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
> > >        at
> > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
> > >        at
> > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
> > >        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> > >        at org.apache.pig.Main.run(Main.java:624)
> > >        at org.apache.pig.Main.main(Main.java:170)
> > >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >        at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > >        at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >        at java.lang.reflect.Method.invoke(Method.java:606)
> > >        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > >
> > >
> > >
> >
> ================================================================================
> > >
> > >
> > > The release notes promise compatibility with Hadoop 2.X. Is this a bug?
> > > Is there a workaround?
> > >
> > > Best regards and thanks for your help.
> > > -- Michael
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
> > > datasyndrome.com
> > >
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message