hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5478) [YARN-4902] Define Java API for generalized & unified scheduling-strategies.
Date Wed, 22 Mar 2017 23:20:41 GMT

     [ https://issues.apache.org/jira/browse/YARN-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wangda Tan updated YARN-5478:
    Attachment: YARN-5478.2.patch

Had discussions with [~vinodkv]/[~asuresh]/[~kkaranasos]/[~subru] offline.

I think now we generally agree that we should stop investing on the old ResourceRequest and
we should move APIs for new features like
- Allocation tag: YARN-6345.
- Affinity/Anti-affinity: YARN-1042.
- Node attributes: YARN-4902.
To the new ResourceRequest.

We will create YARN-4902 branch and put all API-related changes to the branch and upcoming
features like YARN-1042/YARN-6345 can do cherry-pick from YARN-4902.

The basic idea of the patch is to make new/old ResourceRequest can be used in parallel, in
the short term they will provide disjoint sets of features: Old ResourceRequest continues
to provide functionalities to specify capability(resource)/num_containers/priority/locality,
etc. And new ResourceRequest will provide additional features like allocation tag/constraint
specs such as affinity/anti-affinity/node-attribute, etc. And the two will be linked by their
allocation_request_id. And one big advantage of this proposal is completely backward compatible
and no changes need to be made for existing apps.

And slowly we will move more fields from old ResourceRequest to the new one and finally deprecate
the old one.

>From application's perspective, they can specify the new ResourceRequests in both of ApplicationSubmissionContext/AllocationRequest.
There's an on-going discussion about what the protocol should look like. So I didn't include
any changes in ApplicationSubmissionContext/AllocationRequest. Will update the patch once
we reach consensus on that.

For overall API of the patch, please take a look at {{Examples.java}}.

> [YARN-4902] Define Java API for generalized & unified scheduling-strategies.
> ----------------------------------------------------------------------------
>                 Key: YARN-5478
>                 URL: https://issues.apache.org/jira/browse/YARN-5478
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-5478.1.patch, YARN-5478.2.patch, YARN-5478.preliminary-poc.1.patch,
> Define Java API for application to specify generic scheduling requirements described
in YARN-4902 design doc.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message