hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11211) mapreduce.job.classloader.system.classes property behaves differently when the exclusion and inclusion order is different
Date Thu, 20 Nov 2014 19:23:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219825#comment-14219825
] 

Gera Shegalov commented on HADOOP-11211:
----------------------------------------

[~timyitong], sorry for confusion, I did not realize it was your intention to add wording
about the nested classes. The reason for confusion is that patterns with '$' are not supported
and it was the patch MAPREDUCE-6128.v05.patch that tried to add this pattern.

HADOOP-11309 implements matching all nested classes of the class C if the pattern is a single
class C.

If you want to be precise, you can spell out the format of system.classes: comma-separated
list of patterns.
A pattern may define an inclusion (positive):
# A single class 'C' that matches C and transitively all nested classes C$* defined in C
# A package name followed by a '.' (e.g., "com.example.") that matches all classes files from
the package

Exclusion can be defined by a negative pattern -P if P is a positive pattern as defined above.

A class is considered a system class if and only if it matches one of the positive patterns
and none of the negative ones. More formally:

A class is member of the Inclusion set {{I}} if it is matched by one of the positive patterns.
A class is member of the Exclusion set {{E}} if it is matched by one of negative patterns
P. The set of system classes {{S = I \ E}}






> mapreduce.job.classloader.system.classes property behaves differently when the exclusion
and inclusion order is different
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-11211
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11211
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: hudson
>            Reporter: Yitong Zhou
>            Assignee: Yitong Zhou
>             Fix For: hudson
>
>         Attachments: HADOOP-11211.patch
>
>
> If we want to include package foo.bar.* but exclude all sub packages named foo.bar.tar.*
in system classes, configuring "mapreduce.job.classloader.system.classes=foo.bar.,-foo.bar.tar."
won't work. foo.bar.tar will still be pulled in. But if we change the order:
> "mapreduce.job.classloader.system.classes=-foo.bar.tar.,foo.bar.", then it will work.
> This bug is due to the implementation of ApplicationClassLoaser#isSystemClass in hadoop-common,
where we simply return the matching result immediately when the class name hits the first
match (either positive or negative).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message