hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9394) branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing.
Date Fri, 06 Nov 2015 06:49:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993229#comment-14993229
] 

Chris Nauroth commented on HDFS-9394:
-------------------------------------

The easiest way to see this problem is to run the hadoop-hdfs-client tests on branch-2.  Several
of the tests will fail like this:

{code}
Running org.apache.hadoop.fs.TestUrlStreamHandlerFactory
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.2 sec <<< FAILURE!
- in org.apache.hadoop.fs.TestUrlStreamHandlerFactory
testConcurrency(org.apache.hadoop.fs.TestUrlStreamHandlerFactory)  Time elapsed: 0.148 sec
 <<< ERROR!
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.web.HftpFileSystem
not found
	at java.util.ServiceLoader.fail(ServiceLoader.java:231)
	at java.util.ServiceLoader.access$300(ServiceLoader.java:181)
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:365)
	at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
	at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2662)
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2673)
	at org.apache.hadoop.fs.FsUrlStreamHandlerFactory.<init>(FsUrlStreamHandlerFactory.java:62)
	at org.apache.hadoop.fs.FsUrlStreamHandlerFactory.<init>(FsUrlStreamHandlerFactory.java:55)
	at org.apache.hadoop.fs.TestUrlStreamHandlerFactory.singleRun(TestUrlStreamHandlerFactory.java:48)
	at org.apache.hadoop.fs.TestUrlStreamHandlerFactory.testConcurrency(TestUrlStreamHandlerFactory.java:43)
{code}

This also implies that end users wouldn't be successful in depending on hadoop-hdfs-client
only.  They'd still have to get hadoop-hdfs.jar onto the classpath as a workaround.

This is not a problem on trunk, where {{HftpFileSystem}} and {{HsftpFileSystem}} have been
removed.

FYI [~liuml07] and [~wheat9].  Thanks!

> branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because
HftpFileSystem is missing.
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9394
>                 URL: https://issues.apache.org/jira/browse/HDFS-9394
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>            Reporter: Chris Nauroth
>            Priority: Critical
>
> On branch-2, hadoop-hdfs-client contains a {{FileSystem}} service descriptor that lists
{{HftpFileSystem}} and {{HsftpFileSystem}}.  These classes do not reside in hadoop-hdfs-client.
 Instead, they reside in hadoop-hdfs.  If the application has hadoop-hdfs-client.jar on the
classpath, but not hadoop-hdfs.jar, then this can cause a {{ServiceConfigurationError}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message