hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Howard (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10372) Deprecate S3 implementation.
Date Thu, 27 Feb 2014 18:47:22 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13914857#comment-13914857
] 

Michael Howard commented on HADOOP-10372:
-----------------------------------------

Amazon EMR support says that they only uses NativeS3FileSystem within ElasticMapReduce. 
Case 165294761
A: On EMR both s3:// and s3n:// both map to the same implementation and they are using the
same java class "org.apache.hadoop.fs.s3native.NativeS3FileSystem" . This is implemented with
combination of AWS SDK for Java and open source 3rd party code as pointed by my colleague
Fei. You can also confirm this by checking the job configuration XML file on an EMR cluster.
If you have S3 logging enabled on an EMR cluster you can access this configuration file via
s3://<bucket>/<JobFlow-ID>/jobs/job_XXXXXXXXXX_XXXX_conf.xml and you should be
able to see the following lines: 
<property><name>fs.s3n.impl</name><value>org.apache.hadoop.fs.s3native.NativeS3FileSystem</value></property>
<property><name>fs.s3.impl</name><value>org.apache.hadoop.fs.s3native.NativeS3FileSystem</value></property>

> Deprecate S3 implementation.
> ----------------------------
>
>                 Key: HADOOP-10372
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10372
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>            Reporter: Amandeep Khurana
>            Priority: Minor
>
> We encourage users to use the S3N implementation. We should consider deprecating the
older S3 implementation to avoid confusion down the road.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message