pig-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dimitris.iliopou...@gmail.com
Subject Re: Unable to run Pig in MapReduce mode
Date Sat, 02 Apr 2011 01:51:35 GMT
Running the 0.8.0 release with Hadoop 0.20.2 and the rest are pretty standard.
Sent via BlackBerry from T-Mobile

-----Original Message-----
From: Dmitriy Ryaboy <dvryaboy@gmail.com>
Date: Fri, 1 Apr 2011 18:26:34 
To: <user@pig.apache.org>
Reply-To: user@pig.apache.org
Subject: Re: Unable to run Pig in MapReduce mode

Are you running a released version of Pig or something from svn?
Anything interesting about your log setup or classpath we should know about?

D

On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos <
dimitris.iliopoulos@gmail.com> wrote:

> There is a "Fail" link and the error trace that I get from there is as
> follows:
>
> java.io.IOException: Deserialization error:
> org.apache.commons.logging.impl.Log4JLogger;
> org.apache.commons.logging.impl.Log4JLogger; class invalid for
> deserialization
>
>        at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
>        at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217)
>        at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244)
>        at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274)
>        at org.apache.hadoop.mapred.Task.initialize(Task.java:415)
>        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354)
>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.InvalidClassException:
> org.apache.commons.logging.impl.Log4JLogger;
> org.apache.commons.logging.impl.Log4JLogger; class invalid for
> deserialization
>        at
> java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713)
>        at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
>        at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
>        at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
>        at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
>        at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
>        at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
>        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
>        at java.util.LinkedList.readObject(LinkedList.java:964)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
>        at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
>        at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
>        at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
>        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
>        at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
>        ... 6 more
> Caused by: java.io.InvalidClassException:
> org.apache.commons.logging.impl.Log4JLogger; class invalid for
> deserialization
>        at
> java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587)
>        at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
>        at
> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
>        at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
>        ... 23 more
>
>
> On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy <dvryaboy@gmail.com> wrote:
>
> > Is there a line that says "Job Setup: Failed" at the top of the job page?
> > Failed (or "Successful") should be a link, click through that to get the
> > setup log.
> >
> > D
> >
> > On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos <
> > dimitris.iliopoulos@gmail.com> wrote:
> >
> > > Dmitriy,
> > >
> > > The jobtracker does indeed mark the job as failed but the problem is
> that
> > > there are no log files. All map and reduce tasks are flagged as killed
> > and
> > > upon inspection produce no logs. The only logs are the ones I appended
> > > before produced by Pig.
> > >
> > > - Dimitris
> > >
> > > On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
> > wrote:
> > >
> > > > Dimitris,
> > > > You should be able to look at the jobs submitted to your jobtracker
> and
> > > > find
> > > > out what happened to the pig job (the message " Job terminated with
> > > > anomalous status FAILED" leads me to believe there is some job setup
> > > > problem). Do you see any pig job attempts on your jobtracker UI?
> > > >
> > > > D
> > > >
> > > > On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
> > > > dimitris.iliopoulos@gmail.com> wrote:
> > > >
> > > > > Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things
> in
> > > > local
> > > > > mode as well as run fs -* commands from within Grunt running in
> > > MapReduce
> > > > > mode. I can't seem to be able to execute any Pig Latin statement
in
> > > > > MapReduce mode. The fact that I can browse HDFS and access files
> > leads
> > > me
> > > > > to
> > > > > believe that the setup is correct (also verified by the correct
> > > namenode
> > > > > and
> > > > > jobtracker addresses connected to when firing off any pig command).
> I
> > > > don't
> > > > > know why though I can't run anything else. A simple command like
> the
> > > > > following:
> > > > >
> > > > > data = LOAD 'test.csv' USING PigStorage(',');
> > > > > DUMP data;
> > > > >
> > > > > Produces the following stack trace:
> > > > >
> > > > > Pig Stack Trace
> > > > > ---------------
> > > > > ERROR 1066: Unable to open iterator for alias data
> > > > >
> > > > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
> > Unable
> > > to
> > > > > open iterator for alias data
> > > > >        at org.apache.pig.PigServer.openIterator(PigServer.java:754)
> > > > >        at
> > > > >
> > >
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> > > > >        at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> > > > >        at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> > > > >        at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> > > > >        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
> > > > >        at org.apache.pig.Main.run(Main.java:465)
> > > > >        at org.apache.pig.Main.main(Main.java:107)
> > > > > Caused by: java.io.IOException: Job terminated with anomalous
> status
> > > > FAILED
> > > > >        at org.apache.pig.PigServer.openIterator(PigServer.java:744)
> > > > >        ... 7 more
> > > > >
> > > > > Any other more elaborate collection of Pig statements will produce
> > > this:
> > > > >
> > > > > Pig Stack Trace
> > > > > ---------------
> > > > > ERROR 2244: Job failed, hadoop does not return any error message
> > > > >
> > > > > org.apache.pig.backend.executionengine.ExecException: ERROR 2244:
> Job
> > > > > failed, hadoop does not return any error message
> > > > > at
> > > > >
> > >
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> > > > > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> > > > > at org.apache.pig.Main.run(Main.java:510)
> > > > > at org.apache.pig.Main.main(Main.java:107)
> > > > >
> > > > > Where any input file is flagged as unreadable by the following
> error
> > > > > message
> > > > > during execution:
> > > > >
> > > > > Input(s):
> > > > > Failed to read data from "test.csv"
> > > > >
> > > > > All files are accessible in MapReduce mode via cat, ls etc. I might
> > be
> > > > > missing something very obvious here since this is the first time
> I've
> > > > used
> > > > > Pig so any help would be greatly appreciated.
> > > > >
> > > > > - Dimitris
> > > > >
> > > >
> > >
> >
>

Mime
View raw message