hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (PIG-443) Illustrate for the Types branch
Date Thu, 25 Sep 2008 00:29:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634354#action_12634354
] 

pkamath edited comment on PIG-443 at 9/24/08 5:29 PM:
-------------------------------------------------------------

- A couple of unit tests - TestLogicalPlanBuilder and TestLogToPhyCompiler fail with the patch.

 - The following script fails at the last illustrate.
{code}
a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name, age, gpa);
b = load '/user/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
c = filter a by age < 50;
illustrate c;
d = filter b by age < 50;
illustrate d;
e = cogroup c by (name, age), d by (name, age) ;
illustrate e;
f = foreach e generate flatten(c), flatten(d);
illustrate f;
g = group f by registration;
illustrate g;
h = foreach g generate group, SUM(f.d::contributions);
illustrate h;
i = order h by $1;
illustrate i;

{code}

Error:
{noformat}
...
--------------------------------------
| h | group: bytearray | double |
--------------------------------------
| | libertarian | 1697.08 |
| | republican | 1655.32 |
--------------------------------------

10118 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: hdfs://clusterhost
10143 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to map-reduce job tracker at: clusterhost
java.lang.NullPointerException
        at org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:290)
        at org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:458)
        at org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:416)
        at org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:251)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:105)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40)
        at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:69)
        at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:71)
        at org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
        at org.apache.pig.PigServer.getExamples(PigServer.java:627)
        at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:83)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
        at org.apache.pig.Main.main(Main.java:306)

{noformat}

      was (Author: pkamath):
    - A couple of unit tests - TestLogicalPlanBuilder and TestLogToPhyCompiler fail with the
patch.

 - The following script fails at the last illustrate.
{code}
a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name, age, gpa);
b = load '/user/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
c = filter a by age < 50;
illustrate c;
d = filter b by age < 50;
illustrate d;
e = cogroup c by (name, age), d by (name, age) ;
illustrate e;
f = foreach e generate flatten(c), flatten(d);
illustrate f;
g = group f by registration;
illustrate g;
h = foreach g generate group, SUM(f.d::contributions);
illustrate h;
i = order h by $1;
illustrate i;

{code}

Error:
{noformat}
...
--------------------------------------
| h | group: bytearray | double |
--------------------------------------
| | libertarian | 1697.08 |
| | republican | 1655.32 |
--------------------------------------

10118 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: hdfs://wilbur20.labs.corp.sp1.yahoo.com:802013
10143 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to map-reduce job tracker at: wilbur20.labs.corp.sp1.yahoo.com:50020
java.lang.NullPointerException
        at org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:290)
        at org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:458)
        at org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:416)
        at org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:251)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:105)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40)
        at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:69)
        at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:71)
        at org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
        at org.apache.pig.PigServer.getExamples(PigServer.java:627)
        at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:83)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
        at org.apache.pig.Main.main(Main.java:306)

{noformat}
  
> Illustrate for the Types branch
> -------------------------------
>
>                 Key: PIG-443
>                 URL: https://issues.apache.org/jira/browse/PIG-443
>             Project: Pig
>          Issue Type: New Feature
>    Affects Versions: types_branch
>            Reporter: Shubham Chopra
>            Assignee: Shubham Chopra
>             Fix For: types_branch
>
>         Attachments: pig_types_22_09.patch, pig_types_23_09.patch, pig_types_24_09.patch
>
>
> This implementation of Illustrate uses visitors to traverse the plans.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message