hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-402) order by on single field with user defined comparator fails
Date Wed, 10 Sep 2008 17:31:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Shravan Matthur Narayanamurthy updated PIG-402:

    Status: Patch Available  (was: Open)

The solution I am providing is to wrap the key inside a tuple whenever a user defined comparison
func is used. For that I have the following in the patch.

1) Created a new Mapper class MapWithComparator in PigMapReduce which will be used whenever
a u.d. comparator is used. The assumuption is that keyType and keyClass will be appropriately
set to Tuple and the collect here wraps the key in a Tuple. This was done to avoid an if branch
in the earlier Mapper class.

2) JobControlCompiler: To meet the assumptions in 1 above, the changes to job control compiler
ensures consistency

3) PigMapBase: Incidental. Introduced a tuple factory instance into the base class.

4) TestEvalPipeline: Added a new unit test to test Sort with UDF.

> order by on single field with user defined comparator fails
> -----------------------------------------------------------
>                 Key: PIG-402
>                 URL: https://issues.apache.org/jira/browse/PIG-402
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Shravan Matthur Narayanamurthy
>             Fix For: types_branch
>         Attachments: 402.patch
> register udf.jar;
> a = load 'data';
> c = order a by $0 using MyOrderUDF();
> store c into 'out',

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message