hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yan Zhou (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1648) Split combination may return too many block locations to map/reduce framework
Date Sat, 25 Sep 2010 00:21:33 GMT
Split combination may return too many block locations to map/reduce framework
-----------------------------------------------------------------------------

                 Key: PIG-1648
                 URL: https://issues.apache.org/jira/browse/PIG-1648
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.8.0
            Reporter: Yan Zhou
            Assignee: Yan Zhou
             Fix For: 0.8.0


For instance, if a small split has block locations h1, h2 and h3; another small split has
h1, h3, h4. After combination, the composite split contains 4 block locations. If the number
of component splits is big, then the number of block locations could be big too. In fact,
the  number of block locations serves as a hint to M/R as the best hosts this composite split
should be run on so the list should contain a short list, say 5, of the hosts that contain
the most data in this composite split.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message