pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xianyu (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1812) Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Date Thu, 20 Jan 2011 02:06:44 GMT
Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
--------------------------------------------

                 Key: PIG-1812
                 URL: https://issues.apache.org/jira/browse/PIG-1812
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.8.0, 0.7.0
         Environment: RHEL, Pig 0.8.0
            Reporter: xianyu


Hi, 
I have the following input files:

pkg.txt

a       3       {(123,1.0),(236,2.0)}
a       3       {(236,1.0)}

model.txt

a       123     2       0.33
a       236     2       0.5

My script is listed below:

A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray,
wht:float)});

M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);

B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);

B = distinct B;

H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);

H2 = foreach H1 {
        I = order M by ts;
        J = order B by ts;
        generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts
as tsmap;
}

dump H2;

When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
1 time(s)" and pig error log as below:

Pig Stack Trace

---------------

ERROR 2043: Unexpected error during execution.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for
alias H2
        at org.apache.pig.PigServer.openIterator(PigServer.java:764)
        at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
        at org.apache.pig.Main.run(Main.java:500)
        at org.apache.pig.Main.main(Main.java:107)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
        at org.apache.pig.PigServer.storeEx(PigServer.java:888)
        at org.apache.pig.PigServer.store(PigServer.java:826)
        at org.apache.pig.PigServer.openIterator(PigServer.java:738)
        ... 7 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error
during execution.
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
        at org.apache.pig.PigServer.storeEx(PigServer.java:884)
        ... 9 more
Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad
cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
        at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
        at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
        at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
        ... 11 more

But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;"  this script
can run smoothly. I have also tried other reducer side operations like ORDER, it seems that
they will also trigger above error. This is really very confusing.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message