hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6613) Update json validation for new native services providers
Date Wed, 24 May 2017 18:02:04 GMT

     [ https://issues.apache.org/jira/browse/YARN-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Billie Rinaldi updated YARN-6613:
    Attachment: YARN-6613-yarn-native-services.005.patch

Thanks very much for the review, [~jianhe]!

bq. remove the unused method AbstractClientProvider#processClientOperation
This will be used in SliderClient doClientInstall, but it's commented out right now.

bq. Application configurations are merged into components before persisting, this will increase
app json file size. For hdfs, it won't be a problem though. for zk that's relatively sensitive
to file size, may be an issue. Any reason need to resolve it before persisting?
The problem is for external applications. The external components are read and added to the
current application, and these are supposed to use the external application's global configuration.
Since we don't have multiple global configurations per app, the external application configurations
need to have their external global configurations merged right away. So the easiest way to
handle it is to do all the merging before persisting, instead of having some resolution performed
before persisting and some resolution performed after. It's also easier to test that the resulting
Application will be correct if all the resolution is performed at once.

Previously Slider also allowed configuration properties to be overridden for external components,
which was pretty useful. For example, maybe you want to use the external application hbase,
but change the number of region servers, or tweak the configuration properties. In my new
patch, I will allow this as well.

bq. In actionStart, why is it required to write back to hdfs?
It's because the applicationId is part of the Application now. If the Application is not written
back to HDFS, the id will be wrong.

> Update json validation for new native services providers
> --------------------------------------------------------
>                 Key: YARN-6613
>                 URL: https://issues.apache.org/jira/browse/YARN-6613
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>             Fix For: yarn-native-services
>         Attachments: YARN-6613-yarn-native-services.001.patch, YARN-6613-yarn-native-services.002.patch,
YARN-6613-yarn-native-services.003.patch, YARN-6613-yarn-native-services.004.patch, YARN-6613-yarn-native-services.005.patch
> YARN-6160 started some work enabling different validation for each native services provider.
The validation done in ServiceApiUtil#validateApplicationPayload needs to updated accordingly.
This validation should also be updated to handle the APPLICATION artifact type, which does
not have an associated provider.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message