hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1700) User supplied dependencies may conflict with MapReduce system JARs
Date Tue, 04 Sep 2012 18:35:09 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447911#comment-13447911

Arun C Murthy commented on MAPREDUCE-1700:

Tom, I don't understand specific advantages of OSGI or Felix, so please pardon some of my

However, with MR being an application in YARN (see MAPREDUCE-4421) we can just add user jars
in front of the classpath for the tasks (we already allow it). This isn't the same Map/Reduce
child inherits the TT classpath problem in MR1 (actually even in MR1 you can put child jars
ahead in the classpath for a long while now). Given this, do we need to bring in OSGI or Felix,
what do else do they provide? Thanks.
> User supplied dependencies may conflict with MapReduce system JARs
> ------------------------------------------------------------------
>                 Key: MAPREDUCE-1700
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1700
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>            Reporter: Tom White
>            Assignee: Tom White
>         Attachments: MAPREDUCE-1700.patch, MAPREDUCE-1700.patch
> If user code has a dependency on a version of a JAR that is different to the one that
happens to be used by Hadoop, then it may not work correctly. This happened with user code
using a different version of Avro, as reported [here|https://issues.apache.org/jira/browse/AVRO-493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852081#action_12852081].
> The problem is analogous to the one that application servers have with WAR loading. Using
a specialized classloader in the Child JVM is probably the way to solve this.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message