hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Bug in LocalJobRunner?
Date Thu, 21 Mar 2013 12:58:32 GMT
Hi Alex,

This seems to make sense. Could you post your patch with a possible
(even if detached) test case?

On Thu, Mar 21, 2013 at 6:14 AM, Alex Baranau <alex.baranov.v@gmail.com> wrote:
> Hi!
>
> In a test I tried to run a job which is in separate jar with
> MiniMRYarnCluster and got ClassNotFoundException for the Mapper class that
> is in that jar.
>
> I went a bit deeper and found possible bug in LocalJobRunner:
>
> currently:
>
>       this.job = new JobConf(localJobFile);
>
>       // Job (the current object) is a Thread, so we wrap its class loader.
>       if (localDistributedCacheManager.hasLocalClasspaths()) {
>         setContextClassLoader(localDistributedCacheManager.makeClassLoader(
>                 getContextClassLoader()));
>       }
>
> should be:
>
>       this.job = new JobConf(localJobFile);
>       // Job (the current object) is a Thread, so we wrap its class loader.
>       if (localDistributedCacheManager.hasLocalClasspaths()) {
>         ClassLoader classLoader =
> localDistributedCacheManager.makeClassLoader(getContextClassLoader());
>         setContextClassLoader(classLoader);
>         this.job.setClassLoader(classLoader);
>       }
>
> I.e. we need to set classloader for job configuration so that it can load
> classes from the jar.
>
> If the above makes sense I will file JIRA with patch, otherwise, what am I
> missing?
>
>
> Thank you,
> Alex Baranau



-- 
Harsh J

Mime
View raw message