hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2464) Provide Hadoop as a local resource (on HDFS) which can be used by other projects
Date Mon, 08 Dec 2014 16:17:12 GMT

     [ https://issues.apache.org/jira/browse/YARN-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Junping Du updated YARN-2464:
-----------------------------
    Summary: Provide Hadoop as a local resource (on HDFS) which can be used by other projects
 (was: Provide Hadoop as a local resource (on HDFS) which can be used by other projcets)

> Provide Hadoop as a local resource (on HDFS) which can be used by other projects
> --------------------------------------------------------------------------------
>
>                 Key: YARN-2464
>                 URL: https://issues.apache.org/jira/browse/YARN-2464
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Junping Du
>
> DEFAULT_YARN_APPLICATION_CLASSPATH are used by YARN projects to setup their AM / task
classpaths if they have a dependency on Hadoop libraries.
> It'll be useful to provide similar access to a Hadoop tarball (Hadoop libs, native libraries)
etc, which could be used instead - for applications which do not want to rely upon Hadoop
versions from a cluster node. This would also require functionality to update the classpath/env
for the apps based on the structure of the tar.
> As an example, MR has support for a full tar (for rolling upgrades). Similarly, Tez ships
hadoop libraries along with it's build. I'm not sure about the Spark / Storm / HBase model
for this - but using a common copy instead of everyone localizing Hadoop libraries would be
useful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message