Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of slitzferrari@gmail.com
 designates 74.125.44.30 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:references;
        b=Anlw4rdQoe16DSSplT30SD6rTxJrIBWDqh2sx25bXq/7KMBBbJz636FU9W2gnRossI
         7DhyaU8laHNPs3fOz6fY4DALJk/99c0j53InGvh26mI1il8pVoq678IoT785uzrKRWex
         RB1qscyfiVXYjjp7miBL2B+THfpq2BUXfKSnY=
Message-ID: <f12e94740807161803q17f36c1cxbb702b72a0a5a951@mail.gmail.com>
Date: Thu, 17 Jul 2008 02:03:02 +0100
From: slitz <slitzferrari@gmail.com>
To: core-user@hadoop.apache.org
Subject: Re: Namenode Exceptions with S3
In-Reply-To: <b23e01b50807111446l5d772126ge3f218dd3012b5ac@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_39617_12448080.1216256582293"
References: <5dd5b7e40807011634g43ff1351l57399fa323853f09@mail.gmail.com>
	 <b23e01b50807101355l4ec18f97n2347b86964a1918b@mail.gmail.com>
	 <5dd5b7e40807101406v4599365duf26aafae07db7cd1@mail.gmail.com>
	 <b23e01b50807110421p5f0917f1xce80618400e22257@mail.gmail.com>
	 <5dd5b7e40807110831u5cef33abr1cdfec6f04c9d4d2@mail.gmail.com>
	 <f12e94740807111309j242e106duec1176e9a6a51c88@mail.gmail.com>
	 <b23e01b50807111446l5d772126ge3f218dd3012b5ac@mail.gmail.com>

------=_Part_39617_12448080.1216256582293
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Sorry for the time taken to respond, i've been doing some tests on this.
Your workaround worked like a charm, thank you :) now i'm able to fetch the
data from S3 process using HDFS and put the results in S3.

about the a) problem that i mentioned in my previous email, now i understood
the error, i was starting namenode and datanodes and changing
fs.default.name to s3://bucket/ after that, now i understand why it doesn't
work.

Thank you *very* much for your help, now i can use EC2 and S3 :)

slitz

On Fri, Jul 11, 2008 at 10:46 PM, Tom White <tom.e.white@gmail.com> wrote:

> On Fri, Jul 11, 2008 at 9:09 PM, slitz <slitzferrari@gmail.com> wrote:
> > a) Use S3 only, without HDFS and configuring fs.default.name as
> s3://bucket
> >  -> PROBLEM: we are getting ERROR org.apache.hadoop.dfs.NameNode:
> > java.lang.RuntimeException: Not a host:port pair: XXXXX
>
> What command are you using to start Hadoop?
>
> > b) Use HDFS as the default FS, specifying S3 only as input for the first
> Job
> > and output for the last(assuming one has multiple jobs on same data)
> >  -> PROBLEM: https://issues.apache.org/jira/browse/HADOOP-3733
>
> Yes, this is a problem. I've added a comment to the Jira description
> describing a workaround.
>
> Tom
>

------=_Part_39617_12448080.1216256582293--