hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weiwei Yang <cheersy...@hotmail.com>
Subject Re: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk
Date Tue, 05 Dec 2017 02:29:41 GMT
+1 (non-binding)
Thanks for getting this done Sunil.

--
Weiwei

On 5 Dec 2017, 4:06 AM +0800, Eric Payne <erichadoop-1@yahoo.com.invalid>, wrote:
+1. Thanks Sunil for the work on this branch.
Eric

From: Sunil G <sunilg@apache.org
To: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>; Hdfs-dev <hdfs-dev@hadoop.apache.org>;
Hadoop Common <common-dev@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org
Sent: Wednesday, November 29, 2017 7:56 PM
Subject: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881)
to trunk

Hi All,


Based on the discussion at [1], I'd like to start a vote to merge feature
branch

YARN-5881 to trunk. Vote will run for 7 days, ending Wednesday Dec 6 at
6:00PM PDT.


This branch adds support to configure queue capacity as absolute resource in

capacity scheduler. This will help admins who want fine control of
resources of queues.


Feature development is done at YARN-5881 [2], jenkins build is here
(YARN-7510 [3]).

All required tasks for this feature are committed. This feature changes
RM’s Capacity Scheduler only,

and we did extensive tests for the feature in the last couple of months
including performance tests.


Key points:

- The feature is turned off by default, and have to configure absolute
resource to enable same.

- Detailed documentation about how to use this feature is done as part of
[4].

- No major performance degradation is observed with this branch work. SLS
and UT performance

tests are done.


There were 11 subtasks completed for this feature.


Huge thanks to everyone who helped with reviews, commits, guidance, and

technical discussion/design, including Wangda Tan, Vinod Vavilapalli,
Rohith Sharma K S, Eric Payne .


[1] :
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4sBvN_n_tV5GHsbJ3YeyJP%2BP4Q%40mail.gmail.com%3E

[2] : https://issues.apache.org/jira/browse/YARN-5881

[3] : https://issues.apache.org/jira/browse/YARN-7510

[4] : https://issues.apache.org/jira/browse/YARN-7533


Regards

Sunil and Wangda

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message