hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1824) Make Windows client work with Linux/Unix cluster
Date Wed, 12 Mar 2014 11:24:44 GMT

    [ https://issues.apache.org/jira/browse/YARN-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931653#comment-13931653
] 

Steve Loughran commented on YARN-1824:
--------------------------------------

# YARN-1565 relates to this a bit, as it proposes the RM actually serving up some of this
information (e.g. lib path). 
# the env var expansion strategy is going to complicate the {{yarn.application.classpath}}
value

{code}
  <property>
    <name>yarn.application.classpath</name>
    <value>
      /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*
    </value>
  </property>
  
{code}

What will change to -something like

{code}
  <property>
    <value>
      {{/etc/hadoop/conf,/usr/lib/hadoop/*}}{{/usr/lib/hadoop/lib/*}}{{/usr/lib/hadoop-hdfs/*}}
{{/usr/lib/hadoop-hdfs/lib/*}}{{,/usr/lib/hadoop-yarn/*}}{{/usr/lib/hadoop-yarn/lib/*}} {{,/usr/lib/hadoop-mapreduce/*}}{{/usr/lib/hadoop-mapreduce/lib/}}*
    </value>
{code}

If this is the case, whitespace between braces should be stripped so it's easier to lay out
paths.

I'd also like to see, in the absence of YARN-1565, the ability to inject the value of {{yarn.application.classpath}}
which the RM knows about. Because today the clients need to know it, and if you get it wrong
you get meaningless errors that are v. hard to track down. 

FWIW, I think in Ant we just let apps use : or , as separators, and just sort it out, with
some special handling for the {{C:/lib/java}} syntax.



> Make Windows client work with Linux/Unix cluster
> ------------------------------------------------
>
>                 Key: YARN-1824
>                 URL: https://issues.apache.org/jira/browse/YARN-1824
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.4.0
>            Reporter: Jian He
>            Assignee: Jian He
>             Fix For: 2.4.0
>
>         Attachments: YARN-1824.1.patch, YARN-1824.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message