hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4421) Remove dependency on deployed MR jars
Date Mon, 29 Jul 2013 21:59:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13723033#comment-13723033
] 

Sandy Ryza commented on MAPREDUCE-4421:
---------------------------------------

bq. Yes, it does if it can. It depends upon the visibility of the item being localized. If
it's PUBLIC the resource will be cached and reused among all users and all jobs. If PRIVATE
the resource will be cached only per-user but reused between jobs for that user. If APPLICATION
then it will only be localized for a single job. See LocalResourceVisibility and ClientDistributedCacheManager.determineCacheVisibilities
for some details.
Ok awesome.

bq. The javadoc is correct in that even for the APPLICATION...
Makes sense.  Could still be a little more clear, but I'll take my complaints somewhere else.

I think my only other question is about the config name.  Should it not be mapreduce.job.framework.path
to make it consistent with other MR configs?
                
> Remove dependency on deployed MR jars
> -------------------------------------
>
>                 Key: MAPREDUCE-4421
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4421
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Arun C Murthy
>            Assignee: Jason Lowe
>         Attachments: MAPREDUCE-4421.patch, MAPREDUCE-4421.patch
>
>
> Currently MR AM depends on MR jars being deployed on all nodes via implicit dependency
on YARN_APPLICATION_CLASSPATH. 
> We should stop adding mapreduce jars to YARN_APPLICATION_CLASSPATH and, probably, just
rely on adding a shaded MR jar along with job.jar to the dist-cache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message