hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-11263) NativeS3FileSystem doesn't work with hadoop-client
Date Mon, 03 Nov 2014 18:42:33 GMT
Sangjin Lee created HADOOP-11263:
------------------------------------

             Summary: NativeS3FileSystem doesn't work with hadoop-client
                 Key: HADOOP-11263
                 URL: https://issues.apache.org/jira/browse/HADOOP-11263
             Project: Hadoop Common
          Issue Type: Bug
    Affects Versions: 2.4.0
            Reporter: Sangjin Lee


When you start using the NativeS3FileSystem (which is in hadoop-common) based on the hadoop-client
set of jars, it fails with a ClassNotFoundException:

{noformat}
Caused by: java.lang.NoClassDefFoundError: org/jets3t/service/ServiceException
	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.createDefaultStore(NativeS3FileSystem.java:280)
	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:270)
	at com.twitter.twadoop.util.hadoop.NativeS3FileSystemWrapper.initialize(NativeS3FileSystemWrapper.java:34)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2438)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2472)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2454)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:384)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
{noformat}

NativeS3FileSystem depends on a library called jets3t, which is not found in the hadoop-client
build. It turns out that this library was specifically excluded in the hadoop-client pom.xml:

{noformat}
        <exclusion>
          <groupId>net.java.dev.jets3t</groupId>
          <artifactId>jets3t</artifactId>
        </exclusion>
{noformat}

This strikes me as an issue, as a component that's part of hadoop-common cannot run with a
hadoop-client build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message