hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "AmazonS3" by TomWhite
Date Thu, 11 Jan 2007 09:12:31 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by TomWhite:

The comment on the change is:
Update following fixes in 0.10.1

  or as a convenient repository for data input to and output from Map/Reduce. In the second
  HDFS is still used for the Map/Reduce phase.
- S3 support was introduced in Hadoop 0.10 ([http://issues.apache.org/jira/browse/HADOOP-574
+ S3 support was introduced in Hadoop 0.10.0 ([http://issues.apache.org/jira/browse/HADOOP-574
- but it needs the patch in [http://issues.apache.org/jira/browse/HADOOP-857 HADOOP-857] to
work properly.
+ but this had a few bugs so you should use Hadoop 0.10.1 or later.
  The patch in [https://issues.apache.org/jira/browse/HADOOP-862 HADOOP-862] makes S3 work
with the
  Hadoop CopyFile tool.
- (Hopefully these patches will be integrated in the next release.)
  = Setting up hadoop to use S3 as a replacement for HDFS =
@@ -75, +74 @@

  the `bin/hadoop distcp` tool. Then once the Map/Reduce job is complete the output is copied
to S3
  as input to a further job, or retrieved as a final result.
- [More instruction will be added after [https://issues.apache.org/jira/browse/HADOOP-862
HADOOP-862] is complete.]
+ [More instructions will be added after [https://issues.apache.org/jira/browse/HADOOP-862
HADOOP-862] is complete.]

View raw message