hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5751) MR app master fails to start in some cases if mapreduce.job.classloader is true
Date Tue, 11 Feb 2014 16:39:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897994#comment-13897994
] 

Sangjin Lee commented on MAPREDUCE-5751:
----------------------------------------

Yes, I think org.mortbay. should be added to the system classes by default. In fact, I think
I'll do both changes: (1) adding org.mortbay. to the system classes, and (2) delay setting
the classloaders until just before startJobs(). I think (2) still has value in case any other
library that comes between these points happens to use the thread context classloader.

I'll submit a patch shortly.

> MR app master fails to start in some cases if mapreduce.job.classloader is true
> -------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5751
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>
> If mapreduce.job.classloader is set to true, and the MR client includes a jetty jar in
its libjars or job jar, the MR app master fails to start. A typical stack trace we get is
as follows:
> {noformat}
> java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration cannot be
cast to org.mortbay.jetty.webapp.Configuration
> 	at org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890)
> 	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462)
> 	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> 	at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> 	at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
> 	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> 	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> 	at org.mortbay.jetty.Server.doStart(Server.java:224)
> 	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> 	at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676)
> 	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208)
> 	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151)
> 	at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> 	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040)
> 	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
> 	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303)
> 	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259)
> {noformat}
> This happens because as part of the MR app master start the jetty classes are loaded
normally through the app classloader, but WebAppContext tries to load the specific Configuration
class via the thread context classloader (which had been set to the user job classloader).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message