hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yanbo Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
Date Thu, 22 Nov 2012 07:36:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502624#comment-13502624
] 

Yanbo Liang commented on HADOOP-9041:
-------------------------------------

As my tracing the process of running test case in maven+junit environment, I have found that
in this environment the ServiceLoader will directly load service providers from the file "target/class/META-INF/services/org.apache.hadoop.fs.FileSystem"
rather than load from the jar package. Because much resource would be loaded from this directory
"target/class/", so the classloader will reuse this URL and then reuse the URL handler for
the scheme of "file". ServiceLoader will call classloader to load resource from the file in
this directory, then the loading will not use the the handler we just register. Due to not
use the handler we just register in FsUrlStreamHandlerFactory, the reusing handler will work
well and not fall into the infinite loop.
So this bug may be difficult to reproduce in junit environment. But in regular development,
it will be dangerous and hidden deep. Until here I think my patch can resolve this problem
and looking forward any comments. 
                
> FileSystem initialization can go into infinite loop
> ---------------------------------------------------
>
>                 Key: HADOOP-9041
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9041
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.0.2-alpha
>            Reporter: Radim Kolar
>            Assignee: Yanbo Liang
>            Priority: Critical
>         Attachments: fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, HADOOP-9041.patch
>
>
> More information is there: https://jira.springsource.org/browse/SHDP-111
> Referenced source code from example is: https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java
> from isolating that cause it looks like if you register: org.apache.hadoop.fs.FsUrlStreamHandlerFactory
before calling FileSystem.loadFileSystems() then it goes into infinite loop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message