hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Map/reduce with input files on S3
Date Tue, 25 Mar 2008 22:05:31 GMT
I don't have the direct answer, but you could also copy the data from S3 to local EC2 disk
and run from there.  The transfer between S3 and EC2 is free.

Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

----- Original Message ----
From: Prasan Ary <voicesnthedark@yahoo.com>
To: hadoop <core-user@hadoop.apache.org>
Sent: Tuesday, March 25, 2008 4:07:15 PM
Subject: Map/reduce with input files on S3

I am running hadoop on EC2. I want to run a jar MR application on EC2 such that input and
output files are on S3.
  I configured hadoop-site.xml so that  fs.default.name property points to my s3 bucket with
all required identifications  (eg; s3://<ID>:<secret key>@<bucket> ). I
created an input directory in this bucket and put an input file in this directory. Then I
restarted hadoop so that the new configuration takes into effect.
  When I try to run the jar file now, I get the message "Hook previously registered"  and
the application dies.
  Any Idea what might have gone wrong?

Never miss a thing.   Make Yahoo your homepage.

View raw message