hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7457) Delay scheduling should be an individual policy instead of part of scheduler implementation
Date Fri, 15 Dec 2017 18:11:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292953#comment-16292953

Konstantinos Karanasos commented on YARN-7457:

bq. I would prefer to ditch missed opportunities option.
[~leftnoteasy], this is the part I was missing.
I agree that time-based delay is better for end-users and that we could eventually get rid
of missed opportunities.
But I don't think we can get rid of them soon, there are too many applications configured
based on them.

Given that we should have both time and missed opportunities (at least for some time), I believe
we should not let one application specify missed opportunities and the other time-based delays.
That's why I was suggesting to have a cluster-wide parameter that will specify the delay implementation,
i.e., whether we want time-based or missed opportunities. And then let applications give a
number for configuring their delay, which will mean either seconds or missed opportunities,
depending on the cluster-wide parameter for the implementation.

> Delay scheduling should be an individual policy instead of part of scheduler implementation
> -------------------------------------------------------------------------------------------
>                 Key: YARN-7457
>                 URL: https://issues.apache.org/jira/browse/YARN-7457
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Tao Yang
> Currently, different schedulers have slightly different delay scheduling implementations.
Ideally we should make delay scheduling independent from scheduler implementation. Benefits
of doing this:
> 1) Applications can choose which delay scheduling policy to use, it could be time-based
/ missed-opportunistic-based or whatever new delay scheduling policy supported by the cluster.
Now it is global config of scheduler.
> 2) Make scheduler implementations simpler and reusable.
> h2. {color:red}Running design doc: https://docs.google.com/document/d/1rY-CJPLbGk3Xj_8sxre61y2YkHJFK8oqKOshro1ZY3A/edit#heading=h.xnzvh9nn283a{color}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message