pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-339) Limit follow cross/union return wrong number of records
Date Tue, 12 Aug 2008 03:48:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alan Gates updated PIG-339:
---------------------------

    Status: Open  (was: Patch Available)

When I run a script like the following:

{code}
a = load 'studenttab10k';
b = load 'votertab10k';
a1 = foreach a generate $0, $1;
b1 = foreach b generate $0, $1;
c = union a1, b1;
d = limit c 100;
store d into 'result';
{code}

I get:
java.io.IOException: Unable to store for alias: 12 [null]
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:1010)
        at org.apache.pig.impl.logicalLayer.LOLimit.visit(LOLimit.java:76)
        at org.apache.pig.impl.logicalLayer.LOLimit.visit(LOLimit.java:10)
        at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:245)
        at org.apache.pig.PigServer.compilePp(PigServer.java:590)
        at org.apache.pig.PigServer.execute(PigServer.java:516)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:265)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:425)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:82)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
        at org.apache.pig.Main.main(Main.java:302)
Caused by: java.lang.NullPointerException


> Limit follow cross/union return wrong number of records
> -------------------------------------------------------
>
>                 Key: PIG-339
>                 URL: https://issues.apache.org/jira/browse/PIG-339
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: types_branch
>            Reporter: Daniel Dai
>             Fix For: types_branch
>
>         Attachments: PIG-339.patch
>
>
> The following script returns double records as expected:
> a = load 'a';
> b = load 'b';
> c = union a, b;
> d = cross a, b;
> e = limit c 100;
> f = limit d 100;
> dump e;   // return double number of records
> dump f;    // return double number of records
> Seems to be the limit operator in reduce plan is not effective.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message