hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1700) User supplied dependencies may conflict with MapReduce system JARs
Date Tue, 04 Sep 2012 19:46:08 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447996#comment-13447996

Steve Loughran commented on MAPREDUCE-1700:


I see where Tom is coming from. Irrespective of how the Hadoop services are deployed, you
need to be able to do things like submit jobs from OSGi containers (e.g Spring & others)
which is what this patch appears to offer. And if Oracle finally commit to OSGi now that Java
8 is being redefined, it'd be good from all clients.

I would like to see a way to support this which doesn't put an OSGi JAR on the classpath of

Tom -is there a way to abstract away OSGi support so that it's optional, even if its a subclass
of JobSubmitter? An {{org.apache.hadoop.mapreduce.osgi.OSGiJobSubmitter}} could override some
new specific protect methods to enable this.
> User supplied dependencies may conflict with MapReduce system JARs
> ------------------------------------------------------------------
>                 Key: MAPREDUCE-1700
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1700
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>            Reporter: Tom White
>            Assignee: Tom White
>         Attachments: MAPREDUCE-1700.patch, MAPREDUCE-1700.patch
> If user code has a dependency on a version of a JAR that is different to the one that
happens to be used by Hadoop, then it may not work correctly. This happened with user code
using a different version of Avro, as reported [here|https://issues.apache.org/jira/browse/AVRO-493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852081#action_12852081].
> The problem is analogous to the one that application servers have with WAR loading. Using
a specialized classloader in the Child JVM is probably the way to solve this.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message