hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6128) Automatic addition of bundled jars to distributed cache
Date Sat, 08 Nov 2014 08:07:34 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gera Shegalov updated MAPREDUCE-6128:
-------------------------------------
    Attachment: MAPREDUCE-6128.v02.patch

Jason, thank you for review.

Added a test, and It helped uncover classloading defects that were hidden by the fact that
jobclient-tests jar is always added to task classpath when running on minicluster. Along the
way I fixed that even a job.jar that is not uploaded via copyJar and hence can have a different
name is correctly added to the classpath.

For easier testing, I introduced another job classloader pattern Klass$ that matches Klass
and all nested classes of Klass



> Automatic addition of bundled jars to distributed cache 
> --------------------------------------------------------
>
>                 Key: MAPREDUCE-6128
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6128
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 2.5.1
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>         Attachments: MAPREDUCE-6128.v01.patch, MAPREDUCE-6128.v02.patch
>
>
> On the client side, JDK adds Class-Path elements from the job jar manifest
> on the classpath. In theory there could be many bundled jars in many directories such
that adding them manually via libjars or similar means to task classpaths is cumbersome. If
this property is enabled, the same jars are added
> to the task classpaths automatically.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message