hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mathieu Poumeyrol (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-202) ComparatorFunc provided to ORDER clause is not always honoured
Date Mon, 14 Apr 2008 16:45:05 GMT

    [ https://issues.apache.org/jira/browse/PIG-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588647#action_12588647
] 

Mathieu Poumeyrol commented on PIG-202:
---------------------------------------

After struggling a bit with the local mode evaluator, I found the read issue. The code I was
looking for was actualy in instantiateFunc() method, and my previous patch (1) a useless and
incomplete duplication of this code.

Instead I tried to make sure instantiateFunc() is called on all EvalSpec nodes prior to execution
attempts. I now run a POVisitor in LocalExecutionEngine.execute() before pp.open(). It looks
much better as the 16 tests from testOrderBy are now working in both local and mapreduce mode.

The patch is not ready yet (the visitor only visit what my tests needs it to visit) and there
is still come cleanup and commenting to do.

Any comment welcome.

By the way, the MapReducePlanCompiler patch is a piece of junk, but I'm still convinced there
is a severe performance issue here.

> ComparatorFunc provided to ORDER clause is not always honoured
> --------------------------------------------------------------
>
>                 Key: PIG-202
>                 URL: https://issues.apache.org/jira/browse/PIG-202
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Mathieu Poumeyrol
>         Attachments: EvalSpec.patch, MapreducePlanCompiler.patch, TestOderBy.patch
>
>
> Specifying a comparator function is acknowledge neither by local implementation, nor
by quartile lookup job.
> Patch coming soon.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message