Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 02D6C174EE for ; Mon, 2 Feb 2015 19:07:38 +0000 (UTC) Received: (qmail 12842 invoked by uid 500); 2 Feb 2015 19:07:35 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 12805 invoked by uid 500); 2 Feb 2015 19:07:35 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 12792 invoked by uid 99); 2 Feb 2015 19:07:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Feb 2015 19:07:35 +0000 Date: Mon, 2 Feb 2015 19:07:35 +0000 (UTC) From: "Anubhav Dhoot (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-3119) Memory limit check need not be enforced unless aggregate usage of all containers is near limit MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14301698#comment-14301698 ] Anubhav Dhoot commented on YARN-3119: ------------------------------------- Thanks [~leftnoteasy], [~aw] for your comments. All of the problems listed are already possible with the current memory management model which uses monitoring and not cgroups. > Memory limit check need not be enforced unless aggregate usage of all containers is near limit > ---------------------------------------------------------------------------------------------- > > Key: YARN-3119 > URL: https://issues.apache.org/jira/browse/YARN-3119 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager > Reporter: Anubhav Dhoot > Assignee: Anubhav Dhoot > Attachments: YARN-3119.prelim.patch > > > Today we kill any container preemptively even if the total usage of containers for that is well within the limit for YARN. Instead if we enforce memory limit only if the total limit of all containers is close to some configurable ratio of overall memory assigned to containers, we can allow for flexibility in container memory usage without adverse effects. This is similar in principle to how cgroups uses soft_limit_in_bytes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)