pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-3751) Generating Splits in Tez should be configurable to AM or client
Date Fri, 07 Feb 2014 01:15:20 GMT
Rohini Palaniswamy created PIG-3751:
---------------------------------------

             Summary: Generating Splits in Tez should be configurable to AM or client
                 Key: PIG-3751
                 URL: https://issues.apache.org/jira/browse/PIG-3751
             Project: Pig
          Issue Type: Sub-task
    Affects Versions: tez-branch
            Reporter: Rohini Palaniswamy
            Assignee: Rohini Palaniswamy


1) TEZ-752 allows setting list of URIs to get delegation tokens. Set that to make Tez get
delegation tokens and calculate input splits on AM
2) Try using Tez Grouping of input splits instead of pig.maxCombinedSplitSize grouping.

Generating splits in AM is supposed to give performance boost. For those case where InputFormat
or OutputFormat get delegation tokens and it is not possible to do that, then have a option
to generate input splits on client. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message