hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3685) NodeManager unnecessarily knows about classpath-jars due to Windows limitations
Date Wed, 27 May 2015 01:26:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560262#comment-14560262
] 

Vinod Kumar Vavilapalli commented on YARN-3685:
-----------------------------------------------

bq. This was true of Linux even before YARN-316, so in that sense, YARN did already have some
classpath logic indirectly.
bq. I was thinking of stuff like yarn.application.classpath, where values are defined in terms
of things like the HADOOP_YARN_HOME and HADOOP_COMMON_HOME environment variables, and those
values might not match the file system layout at the client side.
Hm.. YARN_APPLICATION_CLASSPATH is a simple convenience configuration property that the server
*does not* load, but used by the applications like distributed-shell. And yeah, this convenience
property was never assumed to work with variable installation layouts. Increasingly our apps
are being migrated to a distributed-cache based deployment so as to avoid the layout issue,
so in sum YARN_APPLICATION_CLASSPATH is essentially unused.

> NodeManager unnecessarily knows about classpath-jars due to Windows limitations
> -------------------------------------------------------------------------------
>
>                 Key: YARN-3685
>                 URL: https://issues.apache.org/jira/browse/YARN-3685
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Vinod Kumar Vavilapalli
>
> Found this while looking at cleaning up ContainerExecutor via YARN-3648, making it a
sub-task.
> YARN *should not* know about classpaths. Our original design modeled around this. But
when we added windows suppport, due to classpath issues, we ended up breaking this abstraction
via YARN-316. We should clean this up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message