hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4493) Distibuted Cache Compatability Issues
Date Tue, 31 Jul 2012 16:33:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425903#comment-13425903

Thomas Graves commented on MAPREDUCE-4493:

One minor nit, it might be nice to add a comment by getLocalCacheArchives and getLocalCacheFiles
as to why they were deprecated.

Otherwise looks good.  +1.
> Distibuted Cache Compatability Issues
> -------------------------------------
>                 Key: MAPREDUCE-4493
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4493
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Critical
>         Attachments: MR-4493.txt
> The distributed cache does not work like it does in 1.0.
> mapreduce.job.cache.symlink.create is completely ignored and symlinks are always created
no matter what.  Files and archives without a fragment will also have symlinks created.
> If two cache archives or cache files happen to have the same name, or same symlink fragment
only the last one in the list is localized.
> The localCacheArchives and LocalCacheFiles are not set correctly when these duplicates
happen causing off by one or more errors for anyone trying to use them.
> The reality is that use of symlinking is so common currently that these incompatibilities
are not that likely to show up, but we still need to fix them.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message