hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gergo Repas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6927) Add support for individual resource types requests in MapReduce
Date Thu, 19 Oct 2017 11:39:02 GMT

     [ https://issues.apache.org/jira/browse/YARN-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Gergo Repas updated YARN-6927:
    Attachment: YARN-6927.003.patch

1 - Agreed that it is an overkill - I got rid of the optionals (in two similar situations

2 - Actually it was not ignoring resource type, the reason it worked is because {{Resource.getResourceInformation()}}
was returning the {{ResourceInformation}} directly which I could mutate. Agreed that this
piece should be improved and I added a {{resourceCapability.setResourceInformation(resourceName,
resourceInformation)}} call to set the mutated object back explicitly.

3 - If we did not set the memory from the new config key, we fall back to the am_prefix.resource.mb
- please see the {{boolean memorySet}} variable in {{YARNRunner.generateResourceRequests()}}.
I updated the comment explaining why we're handling the "mb" resource type in a special way.
For am_prefix.resource.cpu-vcores: the old and new config names are the same.

> Add support for individual resource types requests in MapReduce
> ---------------------------------------------------------------
>                 Key: YARN-6927
>                 URL: https://issues.apache.org/jira/browse/YARN-6927
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Daniel Templeton
>            Assignee: Gergo Repas
>         Attachments: YARN-6927.000.patch, YARN-6927.001.patch, YARN-6927.002.patch, YARN-6927.003.patch
> YARN-6504 adds support for resource profiles in MapReduce jobs, but resource profiles
don't give users much flexibility in their resource requests.  To satisfy users' needs, MapReduce
should also allow users to specify arbitrary resource requests.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message