hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6335) Port slider's groovy unit tests to yarn native services
Date Wed, 19 Apr 2017 03:55:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974021#comment-15974021

Jian He commented on YARN-6335:

bq. This normalization is specifically for capping the resource request at the maximum value
allowed by YARN. Slider used to automatically lower the resource request when it was too high
I see, the config name is confusing then. Normalization is always done in yarn no matter whether
client does it or not. IMO, the validation should be done at job submission to always fail
such requests upfront. This is what the API GetNewApplicationResponse#getMaximumResourceCapability
meant for. Silently let the app continue with an invalid resource requests is just making
user confusing later.  An arbitrary large resource request should always not be allowed similar
to a negative resource.

btw, the original implementation in some places like this are also not generic to take other
resource types(e.g. cpus) into account, the org.apache.hadoop.yarn.util.resource.Resources
API should be used for such resource calculations. 

> Port slider's groovy unit tests to yarn native services
> -------------------------------------------------------
>                 Key: YARN-6335
>                 URL: https://issues.apache.org/jira/browse/YARN-6335
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>             Fix For: yarn-native-services
>         Attachments: YARN-6335-yarn-native-services.001.patch, YARN-6335-yarn-native-services.002.patch,
YARN-6335-yarn-native-services.003.patch, YARN-6335-yarn-native-services.004.patch, YARN-6335-yarn-native-services.005.patch,
YARN-6335-yarn-native-services.006.patch, YARN-6335-yarn-native-services.007.patch
> Slider has a lot of useful unit tests implemented in groovy. We could convert these to
Java for YARN native services. This scope of this ticket will include unit / minicluster tests
only and will not include Slider's funtests which require a running cluster.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message