Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Tue, 14 May 2013 18:33:16 +0000 (UTC)
From: "Carlo Curino (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12647127.1368229317133.317887.1368556396729@arcas>
In-Reply-To: <JIRA.12647127.1368229317133@arcas>
References: <JIRA.12647127.1368229317133@arcas>
Subject: [jira] [Commented] (YARN-666) [Umbrella] Support rolling upgrades
 in YARN
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13657337#comment-13657337 ] 

Carlo Curino commented on YARN-666:
-----------------------------------

This seems a very important problem (and a very hard one too). 

Just to toss one more idea around: I think that an HDFS-based shuffle (we are playing around with it and performance are much better than expected) 
could simplify some of the problems, as we could piggyback on datanode decomissioning mechanics to migrate intermediate data out of a node being decomissioned. 
And (a bit obvious) preemption could be a good tool to make the "draining" fast without wasting work (the "administrative" scenarios we mentioned during the conversation in YARN-45). 
                
> [Umbrella] Support rolling upgrades in YARN
> -------------------------------------------
>
>                 Key: YARN-666
>                 URL: https://issues.apache.org/jira/browse/YARN-666
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.0.4-alpha
>            Reporter: Siddharth Seth
>         Attachments: YARN_Rolling_Upgrades.pdf, YARN_Rolling_Upgrades_v2.pdf
>
>
> Jira to track changes required in YARN to allow rolling upgrades, including documentation and possible upgrade routes. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira