hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "AmazonS3" by TobyDiPasquale
Date Thu, 06 Sep 2007 20:15:56 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by TobyDiPasquale:
http://wiki.apache.org/lucene-hadoop/AmazonS3

------------------------------------------------------------------------------
  
  S3 support was introduced in Hadoop 0.10.0 ([http://issues.apache.org/jira/browse/HADOOP-574
HADOOP-574]),
  but this had a few bugs so you should use Hadoop 0.10.1 or later.
+ 
+ = Notes on S3 Support in Hadoop =
+ 
+ The current implementation of support for Amazon S3 in Hadoop is set up such that it uses
S3 as a backing store for a block-based filesystem, very similar to the way HDFS is implemented.
It '''does not''' currently allow you to access any "regular" S3 keys but instead requires
you to set aside an S3 bucket for its block-based format. There is a request in for the ability
to access "regular" S3 files with Hadoop ([https://issues.apache.org/jira/browse/HADOOP-930
HADOOP-930]), but as yet this is unimplemented.
  
  = Setting up hadoop to use S3 as a replacement for HDFS =
  

Mime
View raw message