hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7438) Additional changes to make SchedulingPlacementSet agnostic to ResourceRequest / placement algorithm
Date Sat, 11 Nov 2017 04:34:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16248323#comment-16248323
] 

Wangda Tan commented on YARN-7438:
----------------------------------

[~asuresh], 

Thanks for reviewing the patch.

bq. Am wondering if the ContainerRequest could be more like the AMRMClient#ContainerRequest
- or we can use SchedulingRequest itself (Apologize for the delay in the patch - I will post
it as soon as YARN-6595 is done.)
I'm not sure if I understand this part: did you suggest to use the new SchedulingRequest here
to represent Container's request? My personally preference is to use a wrapper class to make
the inner implementation of requests to be API-agnostic (no matter it is using old RR API
or new SR API) now since this patch is targeted to trunk and YARN-6592 is not merged yet.
And in the future we can switch to SR / keep RR or support both.

Regarding to the wrapper class's name, do you think is it better to call it SchedulingContainerRequest
(or RecoverableContainerRequest). 

bq. Hmm... lets see if can keep it as updateSchedulingRequest itself - like I asked earlier
offline, do you see a case where the old List<ResourceRequest> is more expressive than
the SchedulingRequest ?
I agree that I didn't see old RR is more expressive, however RR might be easier to deal with
MR-like requests. My suggestion to keep both to avoid expensive conversion between RR and
SR. To me, the ideal future is we don't spend too much time to remove the old RR, instead,
we should make the old RR can run as-is and put more effort to innovate the new one. 

I'm happy to revisit this once we get SchedulingRequest added. I'm also prefer to keep the
API as simple as possible.

> Additional changes to make SchedulingPlacementSet agnostic to ResourceRequest / placement
algorithm
> ---------------------------------------------------------------------------------------------------
>
>                 Key: YARN-7438
>                 URL: https://issues.apache.org/jira/browse/YARN-7438
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-7438.001.patch
>
>
> In additional to YARN-6040, we need to make changes to SchedulingPlacementSet to make
it: 
> 1) Agnostic to ResourceRequest (so once we have YARN-6592 merged, we can add new SchedulingPlacementSet
implementation in parallel with LocalitySchedulingPlacementSet to use/manage new requests
API)
> 2) Agnostic to placement algorithm (now it is bind to delayed scheduling, we should update
APIs to make sure new placement algorithms such as complex placement algorithms can be implemented
by using SchedulingPlacementSet).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message