hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4642) MiniMRClientClusterFactory should not use job.setJar()
Date Mon, 10 Sep 2012 16:26:07 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452115#comment-13452115
] 

Alejandro Abdelnur commented on MAPREDUCE-4642:
-----------------------------------------------

Arun, MR1 is just fine. The problem was in the MiniMRClientClusterFactory.java (part of the
adapter in YARN to mimic MR1 API) which was using the setJar() to set up minicluster classes
in the classpath. Thus if the job running against the minicluster was trying to use setJar(),
it would not work. With this fix avoid using setJar(), thus allowing testcases submitting
jobs using setJar() to work as expected. Hope this clarifies.


                
> MiniMRClientClusterFactory should not use job.setJar()
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-4642
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4642
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: test
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>             Fix For: 2.0.2-alpha
>
>         Attachments: MAPREDUCE-4642.patch, MAPREDUCE-4642.patch
>
>
> Currently, MiniMRClientClusterFactory does {{job.setJar(callerJar)}} so that the {{callerJar}}
is added to the cache in MR2.  However, this makes the resulting configuration inconsistent
between MR1 and MR2 as in MR1 the job jar is not set and in MR2 its set to the {{callerJar}}.
 This difference can also cause some tests to fail in Oozie.  We should instead use the {{job.addCacheFile()}}
method.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message