hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From RichardGUO Fei <gladiato...@hotmail.com>
Subject RE: Custom Loadfunc problem!
Date Mon, 26 Oct 2009 08:43:27 GMT

Hi,

 

Btw, my program works with ILLUSTRATE but not with DUMP or STORE. It could be a problem of
the slicer or POLoad? Please enlighten me.

 

Thanks,

Richard
 
> Date: Mon, 26 Oct 2009 11:02:04 +0800
> Subject: Re: Custom Loadfunc problem!
> From: zjffdu@gmail.com
> To: pig-dev@hadoop.apache.org
> 
> Hi Richard,
> 
> I am not sure what's your problem, but I suggest you can debug your program
> in pig's local mode.
> 
> And you can even debug it in pig's mapreduce mode locally
> 
> 
> 
> Jeff zhang
> 
> 
> 2009/10/25 RichardGUO Fei <gladiatorcn@hotmail.com>
> 
> >
> > Hi,
> >
> >
> >
> > I implemented a custom loadfunc class to read data from a database. This
> > class implements both Slicer and LoadFunc. It also generates the slices
> > properly. The problem is that when I perform this query:
> >
> >
> >
> > A = LOAD 'table_name' USING CustomStorage;
> >
> > DUMP A;
> >
> >
> >
> > The PIG gives the following error message:
> >
> > 2009-10-25 23:27:34,308 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 0% complete
> > 2009-10-25 23:27:39,920 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 6017: Execution failed, while processing
> > hdfs://access0:9000/tmp/temp1662058595/tmp-659297059
> > 2009-10-25 23:27:39,920 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> > open iterator for alias A
> > at org.apache.pig.PigServer.openIterator(PigServer.java:475)
> > at
> > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:532)
> > at
> > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:190)
> > at
> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> > at
> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:142)
> > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> > at org.apache.pig.Main.main(Main.java:320)
> > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
> > Unable to store alias A
> > at org.apache.pig.PigServer.store(PigServer.java:530)
> > at org.apache.pig.PigServer.openIterator(PigServer.java:458)
> > ... 6 more
> > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
> > 6017: Execution failed, while processing
> > hdfs://access0:9000/tmp/temp1662058595/tmp-659297059
> > at
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:181)
> > at
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> > at
> > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:773)
> > at org.apache.pig.PigServer.store(PigServer.java:522)
> > ... 7 more
> >
> >
> > I am sure that getLocations returns the correct paths. And when I try to
> > add in more log displays in my custom Slice, it does not show up as it runs
> > in Map/Reduce.
> >
> >
> >
> > How am I supposed to debug this? Is there any way to retrieve more detailed
> > exception from the plan execution?
> >
> >
> >
> > Thanks,
> >
> > Richard
> >
> > _________________________________________________________________
> > 全新 Windows 7:简化您的日常工作。寻找最适合您的 PC。
> > http://www.microsoft.com/china/windows/buy/
 		 	   		  
_________________________________________________________________
全新 Windows 7:寻找最适合您的 PC。了解详情。
http://www.microsoft.com/china/windows/buy/ 
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message