hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11263) NativeS3FileSystem doesn't work with hadoop-client unless jets3t added to classpath
Date Tue, 04 Nov 2014 17:08:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196364#comment-14196364
] 

Sangjin Lee commented on HADOOP-11263:
--------------------------------------

Thanks [~stevel@apache.org] for the explanation. I missed that things look different on the
trunk. Yes, I'm fine with wontfix.

> NativeS3FileSystem doesn't work with hadoop-client unless jets3t added to classpath
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-11263
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11263
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.4.0
>            Reporter: Sangjin Lee
>
> When you start using the NativeS3FileSystem (which is in hadoop-common) based on the
hadoop-client set of jars, it fails with a NoClassDefFoundError:
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/jets3t/service/ServiceException
> 	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.createDefaultStore(NativeS3FileSystem.java:280)
> 	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:270)
> 	at com.twitter.twadoop.util.hadoop.NativeS3FileSystemWrapper.initialize(NativeS3FileSystemWrapper.java:34)
> 	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2438)
> 	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90)
> 	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2472)
> 	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2454)
> 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:384)
> 	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
> {noformat}
> NativeS3FileSystem depends on a library called jets3t, which is not found in the hadoop-client
build. It turns out that this library was specifically excluded in the hadoop-client pom.xml:
> {noformat}
>         <exclusion>
>           <groupId>net.java.dev.jets3t</groupId>
>           <artifactId>jets3t</artifactId>
>         </exclusion>
> {noformat}
> This strikes me as an issue, as a component that's part of hadoop-common cannot run with
a hadoop-client build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message