hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shubham Chopra (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-637) limit with order by is broken in local mode
Date Fri, 06 Feb 2009 18:39:59 GMT

     [ https://issues.apache.org/jira/browse/PIG-637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shubham Chopra updated PIG-637:
-------------------------------

    Attachment: 637.patch

This happens because the optimizer eliminates the limit after a sort and puts an attribute
in POSort/LOSort instead. This attribute is not used in the local mode sorting as this would
adversely affect the MR sorting of the samples.

I have modified the code to avoid that optimization happening when executing in the local
mode. I have also added a couple of test cases that verify the plans in both local and MR
mode.

> limit with order by is broken in local mode
> -------------------------------------------
>
>                 Key: PIG-637
>                 URL: https://issues.apache.org/jira/browse/PIG-637
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
>            Assignee: Shubham Chopra
>         Attachments: 637.patch
>
>
> Shubham, could you take a look.
> The following script when ran in local mode just ignores the limit and outputs the entire
data set:
> a = load 'studenttab10k' as (name, age,gpa);
> b = order a by name;
> c = limit b 10;
> dump c;
> The same script works fine in MR mode

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message