Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 08669F335 for ; Wed, 10 Apr 2013 21:07:17 +0000 (UTC) Received: (qmail 50957 invoked by uid 500); 10 Apr 2013 21:07:16 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 50913 invoked by uid 500); 10 Apr 2013 21:07:16 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 50904 invoked by uid 99); 10 Apr 2013 21:07:16 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Apr 2013 21:07:16 +0000 Date: Wed, 10 Apr 2013 21:07:16 +0000 (UTC) From: "Alejandro Abdelnur (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-45) Scheduler feedback to AM to release containers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628247#comment-13628247 ] Alejandro Abdelnur commented on YARN-45: ---------------------------------------- Nice, the proposed functionality comes quite handy for some stuff I'm working on. Regarding the question on how to model the PreemptRequest, have you thought about the following alternative? * The PreemptRequest would contain only a Set. * PreemptResource has the following properties: a String location, a Resource capability * The PreemptResource location can be ANY, a rack or a node. * The PreemptResource capability is the total capacity that should be released. * The AM, if taking the hint, would release containers that match the location and add up to the PreemptResource capability. By doing this, you give full control to the AM to decide what to release by grouping any containers that match the location. Thoughts? > Scheduler feedback to AM to release containers > ---------------------------------------------- > > Key: YARN-45 > URL: https://issues.apache.org/jira/browse/YARN-45 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager > Reporter: Chris Douglas > Assignee: Carlo Curino > Attachments: YARN-45.patch > > > The ResourceManager strikes a balance between cluster utilization and strict enforcement of resource invariants in the cluster. Individual allocations of containers must be reclaimed- or reserved- to restore the global invariants when cluster load shifts. In some cases, the ApplicationMaster can respond to fluctuations in resource availability without losing the work already completed by that task (MAPREDUCE-4584). Supplying it with this information would be helpful for overall cluster utilization [1]. To this end, we want to establish a protocol for the RM to ask the AM to release containers. > [1] http://research.yahoo.com/files/yl-2012-003.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira