hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4549) Distributed cache conflicts breaks backwards compatability
Date Wed, 15 Aug 2012 12:51:38 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435029#comment-13435029

Hudson commented on MAPREDUCE-4549:

Integrated in Hadoop-Hdfs-0.23-Build #344 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/344/])
    svn merge -c -1369197 Reverting: MAPREDUCE-4503 in branch-0.23 until MAPREDUCE-4549 can
be addressed. (Revision 1372573)

     Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1372573
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java

> Distributed cache conflicts breaks backwards compatability
> ----------------------------------------------------------
>                 Key: MAPREDUCE-4549
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4549
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.3, 2.1.0-alpha, 3.0.0, 2.2.0-alpha
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Critical
> I recently put in MAPREDUCE-4503 which went a bit too far, and broke backwards compatibility
with 1.0 in distribtued cache entries.  This is to change the behavior of the distributed
cache to more closely match that of 1.0.
> In 1.0 when adding in a cache archive link the first link would win (be the one that
was created), not the last one as is the current behavior, when there were conflicts then
all of the others were ignored and just did not get a symlink created, and finally no symlink
was created for archives that had did not have a fragment in the URL.  
> To simulate this behavior after we parse the cache files and cache archives configuration
we should walk through all conflicting links and pick the first link that has a fragment to
win.  If no link has a fragment then it is just the first link wins.  All other conflicting
links will have a warning an the name of the link will be changed to include a UUID.  If the
same file is both in the distributed cache as a cache file and a cache archive we will throw
an exception, for backwards compatibility.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message