hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7250) Update Shared cache client api to use URLs
Date Thu, 28 Sep 2017 22:58:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185066#comment-16185066

Hudson commented on YARN-7250:

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12993 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12993/])
YARN-7250. Update Shared cache client api to use URLs. (ctrezzo: rev c114da5e64d14b1d9e614081c4171ea0391cb1aa)
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/SharedCacheClient.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/SharedCacheClientImpl.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestSharedCacheClientImpl.java

> Update Shared cache client api to use URLs
> ------------------------------------------
>                 Key: YARN-7250
>                 URL: https://issues.apache.org/jira/browse/YARN-7250
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Chris Trezzo
>            Assignee: Chris Trezzo
>            Priority: Minor
>             Fix For: 2.9.0, 3.0.0
>         Attachments: YARN-7250-trunk-001.patch
> We should make the SharedCacheClient use api more consistent with other YARN api methods.
We can do this by doing two things:
> # Update the SharedCacheClient#use api so that it returns a URL instead of a Path. Currently
yarn developers have to convert the path to a URL when creating a LocalResources. It would
be much smoother if they could just use a URL passed to them by the shared cache client.
> # Remove the portion of the client that deals with fragments as this is not consistent
with the rest of YARN. This functionality is bleeding in from the MapReduce layer, which uses
fragments to keep track of destination file names. YARN's api does not use fragments. Instead
 the ContainerLaunchContext expects a Map<String, LocalResource> localResources, where
the strings are the destination file names. We should let the YARN application handle destination
file names however it wants instead of pushing this into the shared cache api. Additionally,
fragments are a clunky way to handle this.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message