hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8105) FilterFileSystem should delegate initialize rather than use super
Date Thu, 23 Feb 2012 04:39:49 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214297#comment-13214297
] 

Eli Collins commented on HADOOP-8105:
-------------------------------------

Here's the particular problem you'll see, we NPE in FileSystem because conf is now null (and
there's no way to set conf because the setConf method is not overridden).

{noformat}
java.lang.NullPointerException
	at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
	at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:330)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1249)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1289)
	at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:512)
	at org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:207)
	at org.apache.hadoop.fs.ProxyFileSystem.listStatus(ProxyFileSystem.java:174)
	at org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:207)
	at org.apache.hadoop.hive.ql.QTestUtil.cleanUp(QTestUtil.java:385)
	at org.apache.hadoop.hive.ql.QTestUtil.shutdown(QTestUtil.java:255)
{noformat}
                
> FilterFileSystem should delegate initialize rather than use super
> -----------------------------------------------------------------
>
>                 Key: HADOOP-8105
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8105
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.1
>            Reporter: Eli Collins
>            Priority: Blocker
>
> The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides
all methods of FileSystem with versions that pass all requests to the contained file system."
initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply
called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating.
This breaks a number of the Hive tests. Hive's ProxyLocalFileSystem would be able to work
around this problem by calling setConf, but unfortunately FilterFileSystem does not override
that method. We need to make FilterFileSystem call fs.initialize (instead of using super)
and also override setConf.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message