hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7766) Introduce a new config property for YARN Service dependency tarball location
Date Fri, 19 Jan 2018 23:29:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333002#comment-16333002

Jian He commented on YARN-7766:

  public int actionDependency(String destinationFolder, boolean overwrite)
      throws IOException, YarnException {
    String currentUser = RegistryUtils.currentUser();
    LOG.info("Running command as user {}", currentUser);

    Path dependencyLibTarGzip = fs.getDependencyTarGzip(true,
- For above code, feel like we don't need to  pass in a 'true' flag (this avoids the other
chain of caller changes which passes in "false").  We can do special code handling right in
here something like below ?
    if (destinationFolder == null) {
      destinationFolder = String.format(YarnServiceConstants.DEPENDENCY_DIR,
    Path dependencyLibTarGzip = new Path(destinationFolder,
            + YarnServiceConstants.DEPENDENCY_TAR_GZ_FILE_EXT);

> Introduce a new config property for YARN Service dependency tarball location
> ----------------------------------------------------------------------------
>                 Key: YARN-7766
>                 URL: https://issues.apache.org/jira/browse/YARN-7766
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: applications, client, yarn-native-services
>            Reporter: Gour Saha
>            Assignee: Gour Saha
>            Priority: Major
>         Attachments: YARN-7766.001.patch, YARN-7766.002.patch
> Introduce a new config property (something like _yarn.service.framework.path_ in-line
with _mapreduce.application.framework.path_) for YARN Service dependency tarball location.
This will provide flexibility to the user/cluster-admin to upload the dependency tarball
to a location of their choice. If this config property is not set, YARN Service client will
default to uploading all dependency jars from the client-host's classpath for every service
launch request (as it does today).
> Also, accept an optional destination HDFS location for *-enableFastLaunch* command,
to specify the location where user/cluster-admin wants to upload the tarball. If not specified, let's
default it to the location we use today. The cluster-admin still needs to set _yarn.service.framework.path_
to this default location otherwise it will not be used. So the command-line will become something
like this -
> {code:java}
> yarn app -enableFastLaunch [<Destination Folder>]{code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message