Mailing-List: contact issues-help@tez.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@tez.incubator.apache.org
Date: Tue, 6 Aug 2013 06:26:49 +0000 (UTC)
From: "Siddharth Seth (JIRA)" <jira@apache.org>
To: issues@tez.incubator.apache.org
Message-ID: <JIRA.12662032.1375764907072.6907.1375770409582@arcas>
In-Reply-To: <JIRA.12662032.1375764907072@arcas>
References: <JIRA.12662032.1375764907072@arcas>
Subject: [jira] [Commented] (TEZ-344) Add an option to allow node-local only
 scheduling in the TaskScheduler
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/TEZ-344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13730427#comment-13730427 ] 

Siddharth Seth commented on TEZ-344:
------------------------------------

bq. Whats the use case? Can the testing be done with a private build and avoid code change "for now"? We all know there is nothing like "for now" 
Worst case, it's an indefinite "for now" :), but realistically this is something we should fix sooner rather than later - i.e. delaying allocations in the AM scheduler to increase locality. Changes like these tend to get lost in private builds.  Being able to figure out the impact of locality on different environments is a useful performance tool.
YARN would've been an option, but I don't want to be debugging YARN scheduler allocations while working on Tez. YARN white-listing isn't issue free - and should likely come with it's own warning about usage.
                
> Add an option to allow node-local only scheduling in the TaskScheduler
> ----------------------------------------------------------------------
>
>                 Key: TEZ-344
>                 URL: https://issues.apache.org/jira/browse/TEZ-344
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>              Labels: TEZ-0.2.0
>
> This, for now, is primarily to help with testing of Tez on clusters.
> Would have to go in with a warning since this could cause jobs to hang / run for a long time.
> Longer term, this can be enhanced to set limits on how long to wait before assigning non-local tasks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira