hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
Date Thu, 16 Apr 2015 01:42:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497445#comment-14497445
] 

Sangjin Lee commented on MAPREDUCE-6293:
----------------------------------------

Thanks [~jira.shegalov] for the review!

I checked for null there in the abundance of caution because if {{jobClassLoader}} was null
{{setContextClassLoader()}} would also set the thread's context classloader to null. If the
inherited context classloader was not null initially, then it would have an effect of nulling
out that TCCL. So to be safe, we should not be calling {{setContextClassLoader()}} unless
we're truly overriding it, no?

> uberized job fails with the job classloader enabled
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-6293
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am
>    Affects Versions: 2.6.0
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>         Attachments: MAPREDUCE-6293.001.patch
>
>
> An uberized job fails if the job classloader is enabled and the job needs to use the
thread context classloader to load a class. Some example error in the log:
> {quote}
> 2015-03-23 23:28:34,675 INFO [main\] org.apache.hadoop.mapreduce.v2.util.MRApps: Creating
job classloader
> ...
> 2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\] cascading.provider.ServiceLoader:
unable to find service class: cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException: cascading.tuple.hadoop.collect.HadoopTupleMapFactory
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message