tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (TEZ-344) Add an option to allow node-local only scheduling in the TaskScheduler
Date Tue, 06 Aug 2013 05:49:47 GMT

    [ https://issues.apache.org/jira/browse/TEZ-344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13730379#comment-13730379
] 

Bikas Saha edited comment on TEZ-344 at 8/6/13 5:48 AM:
--------------------------------------------------------

Whats the use case? Can the testing be done with a private build and avoid code change "for
now"? We all know there is nothing like "for now" :)
If we want to really support node-local only scheduling then we should start using the relaxLocality
flag exposed by YARN container requests. That would be a long term solution. In fact, we should
stop adding rack and ANY to the container requests explicitly in the AM. The AMRMclient already
does that. The difference between strict and non-strict would then be only the value of the
relaxLocality flag.
                
      was (Author: bikassaha):
    Whats the use case? Can the testing be done with a private build and avoid code change
"for now"? We all know there is nothing like "for now" :)
If we want to really support node-local only scheduling then we should start using the relaxLocality
flag exposed by YARN container requests. That would be a long term solution.
                  
> Add an option to allow node-local only scheduling in the TaskScheduler
> ----------------------------------------------------------------------
>
>                 Key: TEZ-344
>                 URL: https://issues.apache.org/jira/browse/TEZ-344
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>              Labels: TEZ-0.2.0
>
> This, for now, is primarily to help with testing of Tez on clusters.
> Would have to go in with a warning since this could cause jobs to hang / run for a long
time.
> Longer term, this can be enhanced to set limits on how long to wait before assigning
non-local tasks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message