hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4577) Enable aux services to have their own custom classpath/jar file
Date Fri, 22 Apr 2016 00:59:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253112#comment-15253112
] 

Sangjin Lee commented on YARN-4577:
-----------------------------------

Yes I think the POC patch is pretty close to what I had in mind too.

A couple of more minor suggestions:
- I probably wouldn't make {{AuxServiceWithCustomClassLoader}} public. It should be really
visible only to {{AuxServices}}. Package scope should be fine.
- I understand {{callWithCustomClassLoader()}} is bit complicated because it has to support
methods with different signatures. I would simply inline the code (as you are doing with {{service*()}}
methods). Then you don't have to do any reflection business to do this.

Don't forget to test it with a real-life use case! Thanks.

> Enable aux services to have their own custom classpath/jar file
> ---------------------------------------------------------------
>
>                 Key: YARN-4577
>                 URL: https://issues.apache.org/jira/browse/YARN-4577
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.8.0
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-4577.1.patch, YARN-4577.2.patch, YARN-4577.20160119.1.patch,
YARN-4577.20160204.patch, YARN-4577.3.patch, YARN-4577.3.rebase.patch, YARN-4577.4.patch,
YARN-4577.poc.patch
>
>
> Right now, users have to add their jars to the NM classpath directly, thus put them on
the system classloader. But if multiple versions of the plugin are present on the classpath,
there is no control over which version actually gets loaded. Or if there are any conflicts
between the dependencies introduced by the auxiliary service and the NM itself, they can break
the NM, the auxiliary service, or both.
> The solution could be: to instantiate aux services using a classloader that is different
from the system classloader.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message