pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4392) RANK BY fails when default_parallel is greater than cardinality of field being ranked by
Date Thu, 19 Feb 2015 23:24:13 GMT

    [ https://issues.apache.org/jira/browse/PIG-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328303#comment-14328303
] 

Koji Noguchi commented on PIG-4392:
-----------------------------------

> Hmm. It worked for us in hadoop 2.X. So maybe it depends on the hadoop version.
>
Never mind.  PIG-4220 was handling a different case.

> Apart from this, I'm seeing test failures after this jira (on hadoopversion=23).
>
Opened PIG-4426.

> RANK BY fails when default_parallel is greater than cardinality of field being ranked
by
> ----------------------------------------------------------------------------------------
>
>                 Key: PIG-4392
>                 URL: https://issues.apache.org/jira/browse/PIG-4392
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11.1
>            Reporter: Anthony Hsu
>            Assignee: Daniel Dai
>             Fix For: 0.15.0
>
>         Attachments: PIG-4392-1.patch, PIG-4392-2.patch
>
>
> To reproduce:
> {code:title=input.txt}
> 1 2 3
> 4 5 6
> 7 8 9
> {code}
> {code:title=rank.pig}
> set default_parallel 4;
> d = load 'input.txt' using PigStorage(' ') as (a:int, b:int, c:int);
> e = rank d by a;
> dump e;
> {code}
> If {{default_parallel}} is set to {{3}}, the script succeeds. So I'm guessing RANK BY
has issues if the {{default_parallel}} exceeds the cardinality of the field being ranked by.
> I'm seeing this issue with Pig 0.11.1 (which has the PIG-2932 patch applied) and Hadoop
2.3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message