hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5146) application classloader may be used too early to load classes
Date Tue, 23 Apr 2013 14:09:17 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639085#comment-13639085
] 

Hudson commented on MAPREDUCE-5146:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk #1408 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1408/])
    MAPREDUCE-5146. application classloader may be used too early to load classes. Contributed
by Sangjin Lee. (Revision 1470694)

     Result = SUCCESS
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1470694
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java

                
> application classloader may be used too early to load classes
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-5146
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5146
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>            Priority: Minor
>             Fix For: 2.0.5-beta
>
>         Attachments: classloader-test.patch, MAPREDUCE-5146.patch
>
>
> At least in the case of YarnChild, the application classloader is set fairly early (both
in Configuration and as a TCCL). This has an effect of using the application classloader unexpectedly
early.
> There is a fair amount of code that gets invoked between setting the classloader and
executing mapper/reducer task.
> For example, I saw that the application classloader was asked to load a DOM parser class
(com.sun.org.apache.xerces...) as part of initializing the filesystem. Luckily, in most cases
this would be delegated to the parent classloader as the job classpath would not have those
classes.
> However, in general, this behavior carries the risk of loading classes with the app classloader
accidentally, and potentially causing problems such as ClassCastException. Those would turn
into nasty bugs that are hard to fix.
> It would be good to either set the application classloader as late as possible or place
clearer limitations so it loads only the mapper/reducer classes and their dependencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message