tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-393) Handle custom partitioners
Date Tue, 27 Aug 2013 16:47:52 GMT

    [ https://issues.apache.org/jira/browse/TEZ-393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751423#comment-13751423
] 

Rohini Palaniswamy commented on TEZ-393:
----------------------------------------

bq. The decision to enable or disable dynamic reduce concurrency will be set by the user.
So the user can choose to not use it.
  That is good. Most of our hbase users are writing custom partitioners for pig to partition
data based on their region server partitions. Will this disabling option be per vertex as
we will want to turn it off only for those stages of the DAG where the customer partitioner
is applied?  
                
> Handle custom partitioners
> --------------------------
>
>                 Key: TEZ-393
>                 URL: https://issues.apache.org/jira/browse/TEZ-393
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Rohini Palaniswamy
>
>   With dynamic allocation of reducers, cases like range partitioning (for ORDER BY) need
to be handled properly. Also many users have other custom partitioners. So might be good to
have an option to turn of dynamic reducer allocation in case they don't work well with dynamic
allocation. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message