hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-483) PERFORMANCE: different strategies for large and small order bys
Date Thu, 09 Oct 2008 19:06:46 GMT
PERFORMANCE: different strategies for large and small order bys
---------------------------------------------------------------

                 Key: PIG-483
                 URL: https://issues.apache.org/jira/browse/PIG-483
             Project: Pig
          Issue Type: Improvement
    Affects Versions: types_branch
            Reporter: Olga Natkovich
             Fix For: types_branch


Currently pig always does a multi-pass order by where it first determines a distribution for
the keys and then orders in a second pass.  This avoids the necessity of having a single reducer.
 However, in cases where the data is small enough to fit into a single reducer, this is inefficient.
 For small data sets it would be good to realize the small size of the set and do the order
by in a single pass with a single reducer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message