hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-702) minicluster classpath construction requires user to set yarn.is.minicluster in the job conf
Date Mon, 03 Jun 2013 18:13:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13673385#comment-13673385

Jonathan Hsieh commented on YARN-702:

bq. What may be possible is for the MiniMRCluster to copy these configs into the user specified
config. That would be very similar to a blanket copy.

This sounds great.  This would be a method of some sort that we pass in a Confiugration into
that either mutates it?  I don't like the idea of the mr client layer needed to know which
settings to extract from the minimrcluster's config from an encapsulation point of view
> minicluster classpath construction requires user to set yarn.is.minicluster in the job
> -------------------------------------------------------------------------------------------
>                 Key: YARN-702
>                 URL: https://issues.apache.org/jira/browse/YARN-702
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.0.4-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
> YARN-129 improved classpath construction for miniclusters by, when yarn.is.minicluster
is set, adding the current JVM's classpath to the ContainerLaunchContext for the MR AM and
tasks.  An issue with this is that it requires the user to set yarn.is.minicluster on the
mapreduce side in the job conf, if they are not copying to RM conf into the jobconf.
> I think it would be better to bypass the ContainerLaunchContext and instead have the
nodemanager check the property, and if it is true, do the classpath additions there.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message