hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlo Curino (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5385) Add a PriorityAgent in ReservationSystem
Date Tue, 28 Feb 2017 22:19:45 GMT

    [ https://issues.apache.org/jira/browse/YARN-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888967#comment-15888967

Carlo Curino commented on YARN-5385:

[~seanpo03] thanks for addressing some of my comments, we are getting closer, though I still
have some doubts.
Ok on 1-4, and I am fine postponing 5 to separate JIRA (I agree it will require some more
work, and probably API discussions). 

Ok on 6 via switch.

Regarding 7, I think in general we assume that multiple agents can operate on a single plan,
and the "coordination" was happening in the plan (see writelocks there, and double-checking
whether ReservationAllocation fits). I think we should follow the same style of coordination,
thus finding a way to grab locks on the plan for an atomic set of changes. The fact that we
have only one agent per plan, is not strictly required, mostly there because the invocations
are almost static. I suggest to keep the coordination in the plan.  Also even if we do what
you propose, the lock should only kick-in if you have to make room, and not for normal submissions.
[~subru] I would like your opinion on this: What do you think about having an explicit grabbing
and release of the plan writelocks (from the agent side)? In a sense we are providing begin-transaction,
commit type methods.  [~seanpo03] can you do a QPS test for this? this locks worry me a bit.

Ok to do more complex solutions like in suggested in 8 in YARN-6226.

Good on 9.
I like the default semantics for 10.
Good on 11. 

> Add a PriorityAgent in ReservationSystem 
> -----------------------------------------
>                 Key: YARN-5385
>                 URL: https://issues.apache.org/jira/browse/YARN-5385
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler, fairscheduler, resourcemanager
>            Reporter: Sean Po
>            Assignee: Sean Po
>              Labels: oct16-hard
>         Attachments: YARN-5385.v002.patch, YARN-5385.v003.patch, YARN-5385.v004.patch,
YARN-5385.v005.patch, YARN-5385.v006.patch, YARN-5385.v007.patch, YARN-5385.v1.patch
> YARN-5211 proposes adding support for generalized priorities for reservations in the
YARN ReservationSystem. This JIRA is a sub-task to track the addition of a priority agent
to accomplish it. Please refer to the design doc in the parent JIRA for details.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message