hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weiwei Yang <cheersy...@hotmail.com>
Subject Re: YARN SLS improving idea
Date Tue, 14 Aug 2018 00:54:55 GMT
Hi Sichen

Thanks for proposing this, this for sure a nice enhancement for SLS.
For SLS improvements, feel free to create JIRA tickets under https://issues.apache.org/jira/browse/YARN-5065,
that’s the umbrella we track all related items.
For scheduling requests support, there is an existing JIRA and an existing patch in https://issues.apache.org/jira/browse/YARN-8007,
would you like to take a look?

Thanks
--
Weiwei

From: Sunil G <sunilg@apache.org>
Date: Tuesday, August 14, 2018 at 1:31 AM
To: Yufei Gu <flyrain000@gmail.com>
Cc: Daniel Templeton <daniel@cloudera.com>, YANG WEIWEI <cheersyang@hotmail.com>,
"zsc19940506@outlook.com" <zsc19940506@outlook.com>, Hadoop Common <common-dev@hadoop.apache.org>,
Wangda Tan <wheeleast@gmail.com>
Subject: Re: YARN SLS improving idea

Hi Sichen

1. Add input support for the scheduling request format.
Yes. I suppose this change is in SLS end (client to generate requests)
2. Add support for scheduling request resource format in NMSim.
Makes sense.
3. Adding scheduling request support for the Capacity Scheduler(maybe it is already done in
current version).
This support is already there.

Adding to this, major challenge is to specify constrains per app level and verifying results.
I also suggest to cross check the scheduler invariants check support added as per YARN-6547
and see how we can incorporate same to predict o/p

- Sunil


On Mon, Aug 13, 2018 at 10:39 PM Yufei Gu <flyrain000@gmail.com<mailto:flyrain000@gmail.com>>
wrote:
+YANG WEIWEI<mailto:cheersyang@hotmail.com>

Make sense to me from SLS perspective, but I am not familiar with Placement Constraints. Add
WeiWei.

Best,

Yufei

`This is not a contribution`


On Sat, Aug 11, 2018 at 8:57 AM Daniel Templeton <daniel@cloudera.com<mailto:daniel@cloudera.com>>
wrote:
Yufei, Wangda, Sunil, any comments?

Daniel

On 8/11/18 8:48 AM, Sichen Zhao wrote:
> Hi,
> Is there anyone who can reply my ideas?
>
> Best Regards
> Sichen Zhao
>
> ________________________________________
> From: Sichen Zhao <zsc19940506@outlook.com<mailto:zsc19940506@outlook.com>>
> Sent: Friday, August 10, 2018 11:10
> To: Hadoop Common
> Subject: YARN SLS improving idea
>
> Hi,
> I am a developer from AliBaBa China, i recently used SLS for scheduling simulation, SLS
currently supports multidimensional resource input(CPU, mem , other resources: disk). But
SLS can't take scheduling request, which is currently widely used in YARN, as input, so Placement
Constraints and attributes are not supported.
>
> So what i wanna improve the SLS: Add scheduling emulation for scheduling request resource
format.
>
> The specific work is as follows:
> 1. Add input support for the scheduling request format.
> 2. Add support for scheduling request resource format in NMSim.
> 3. Adding scheduling request support for the Capacity Scheduler(maybe it is already done
in current version).
>
> What do you think about my ideas?
>
>
> Best Regards
> Sichen Zhao
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<mailto:common-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: common-dev-help@hadoop.apache.org<mailto:common-dev-help@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<mailto:common-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: common-dev-help@hadoop.apache.org<mailto:common-dev-help@hadoop.apache.org>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message