From yarn-issues-return-161178-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Tue Jan 15 19:38:48 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id C431A180609 for ; Tue, 15 Jan 2019 19:38:47 +0100 (CET) Received: (qmail 47518 invoked by uid 500); 15 Jan 2019 18:38:46 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 47507 invoked by uid 99); 15 Jan 2019 18:38:46 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Jan 2019 18:38:46 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 715F1CB872 for ; Tue, 15 Jan 2019 18:38:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id SvY4jDUhqkiy for ; Tue, 15 Jan 2019 18:38:45 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id E478160CEA for ; Tue, 15 Jan 2019 18:31:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 62D52E006D for ; Tue, 15 Jan 2019 18:31:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1BF8C25629 for ; Tue, 15 Jan 2019 18:31:00 +0000 (UTC) Date: Tue, 15 Jan 2019 18:31:00 +0000 (UTC) From: "Aihua Xu (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-9200) Enable resource configuration of queue capacity for different resources independently MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743284#comment-16743284 ] Aihua Xu commented on YARN-9200: -------------------------------- Thanks [~leftnoteasy] Good to know it's the right direction. :) [~rohithsharma] Let me know if you are actively working on this. > Enable resource configuration of queue capacity for different resources independently > ------------------------------------------------------------------------------------- > > Key: YARN-9200 > URL: https://issues.apache.org/jira/browse/YARN-9200 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler > Affects Versions: 3.1.0 > Reporter: Aihua Xu > Assignee: Aihua Xu > Priority: Major > > In capacity scheduler, currently two resource allocations are supported. 1. percentage allocation for child queues - the child queue gets a defined percentage of the resources for all the resource types; 2. absolute values (YARN-5881) - each resource is configured an absolute values. > Right now we can't mix these case together and it would also very confusing to mix them in one cluster. The second case actually is more targeting toward cloud env. > In a non-cloud env, the ability to configure each resource independently is also useful, but percentage is preferable over absolute value. One thought here is to add the percentage configuration for each resource type on the queue. That would allow us to configure memory bounded queues, or CPU bounded queues. We can also keep backward compatible: each resource type just gets the same percentage if no percentage is configured for individual resource type. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org