Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of phil123@gmail.com designates
 74.125.82.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=UR6iHz95bEIJxWiz4UDf5dYrKGu4KdJVAXTQl0v6ccPvwlEuEi0/v5/XDJiMAID7KB
         uvPa/azKwcqJH/XpTLFJcpWNW4kYq2vVi81KNyQvJFk20IKDYJ+Wud9Ai2HJLO+mFkHc
         /uK6tOYKqJalnugUWDQRuV6eo5V+SEZhY2ks0=
MIME-Version: 1.0
In-Reply-To: <AANLkTinbq6KDPk6XOWPjXSP8uTiDSiafBjJA_6HAoxtf@mail.gmail.com>
References: <830213.91228.qm@web130109.mail.mud.yahoo.com>
 <AANLkTinuDsXyVzevByxEhnczKazBJgr7pdvP2yC7HmvN@mail.gmail.com>
 <AANLkTikn0wtUc-OK6_sBf2=d6aG85YhDP0uTzguQUJYw@mail.gmail.com>
 <AANLkTinbq6KDPk6XOWPjXSP8uTiDSiafBjJA_6HAoxtf@mail.gmail.com>
From: Phil Whelan <phil123@gmail.com>
Date: Tue, 4 Jan 2011 16:29:23 -0800
Message-ID: <AANLkTimde7oCUNcm5=o2twrd26mT3iMUwq3=98hOq_9v@mail.gmail.com>
Subject: Re: Hbase/Hadoop cluster setup on AWS
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Lars,

Thanks for reading the post and the feedback.

Sorry for slow reply. I was running through it again to check. If I do
not enable the SOCKS proxy to the MacOSX Network Settings, then my web
browser cannot see the AWS internal domain names when I browsing the
HDFS.

e.g. http://ip-10-245-121-242.ec2.internal:50075/browseDirectory.jsp?nameno=
deInfoPort=3D50070&dir=3D/

I'm guessing if you do not browse the file-system through the browser.
Otherwise, it's possible I missed a step that then required me to have
to do this.

Cheers,
Phil

On Tue, Jan 4, 2011 at 3:23 PM, Lars George <lars.george@gmail.com> wrote:
> Hi Phil,
>
> Very nice post indeed! Awesome stuff.
>
> One question I have is re: adding the SOCKS proxy to MacOS. I am on
> MacOS too and did not do that but simply ran the hadoop-proxy.sh
> script to set up the tunnel. What is that option needed for?
>
> Thanks,
> Lars
>
>
> On Tue, Jan 4, 2011 at 11:26 PM, Phil Whelan <phil123@gmail.com> wrote:
>> Hi guys,
>>
>> My recent blog post goes through in detail the steps I took to use
>> Whirr to start-up a CDH based Hadoop cluster on AWS. I'd really
>> appreciate any feedback you have on it. I'm planning to write a
>> similar on HBase.
>>
>> http://www.philwhln.com/map-reduce-with-ruby-using-hadoop
>>
>> Thanks,
>> Phil
>>
>> On Tue, Jan 4, 2011 at 12:23 AM, Lars George <lars.george@gmail.com> wro=
te:
>>> Hi Otis,
>>>
>>> It also supports CDH although it does only start Hadoop
>>> (HDFS/MapReduce). I am going to open a JIRA to facilitate the start up
>>> of other services on top of that.
>>>
>>> Lars
>>>
>>> On Tue, Jan 4, 2011 at 4:31 AM, Otis Gospodnetic
>>> <otis_gospodnetic@yahoo.com> wrote:
>>>> Ah, Whirr and Hadoop/HBase on AWS. =A0 Is this only applicable to usin=
g "raw"
>>>> HBase/Hadoop from ASF or can one use Whirr support for HBase/Hadoop de=
ployment
>>>> to AWS with CDH?
>>>>
>>>> Thanks,
>>>> Otis
>>>> ----
>>>> Sematext :: http://sematext.com/ :: Solr - Lucene - Hadoop - HBase
>>>> Hadoop ecosystem search :: http://search-hadoop.com/
>>>>
>>>>
>>>>
>>>> ----- Original Message ----
>>>>> From: Lars George <lars.george@gmail.com>
>>>>> To: user@hbase.apache.org
>>>>> Sent: Mon, January 3, 2011 12:32:11 PM
>>>>> Subject: Re: Hbase/Hadoop cluster setup on AWS
>>>>>
>>>>> Hi H,
>>>>>
>>>>> While you can do that by hand I strongly recommend using Apache =A0Wh=
irr
>>>>> (http://incubator.apache.org/projects/whirr.html) which has =A0Hadoop=
 and
>>>>> (in trunk now) also HBase support, straight from the Apache =A0tarbal=
ls.
>>>>>
>>>>> If you want to set them up manually then you simply spin up N =A0mach=
ines
>>>>> and follow the normal guides for either project to set them =A0up
>>>>> appropriately, so extra magic needed.
>>>>>
>>>>> Lars
>>>>>
>>>>> On Mon, Jan 3, =A02011 at 3:26 PM, h <helwyr@gmail.com> wrote:
>>>>> > Are there =A0any good tutorials on cluster setup on AWS
>>>>> > I'd prefer not to use =A0third-party scripts/frameworks, just a sim=
ple
>>>> sequence
>>>>> > of =A0steps/one-liner commands, using the original hadoop/hbase tar=
balls
>>>>> >
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Cell : +1=A0(778) 233-4935
>> Twitter : http://www.twitter.com/philwhln
>> LinkedIn : http://ca.linkedin.com/in/philwhln
>> Blog : http://www.philwhln.com
>> Skype : philwhelan76
>>
>


--=20
Cell : +1=A0(778) 233-4935
Twitter : http://www.twitter.com/philwhln
LinkedIn : http://ca.linkedin.com/in/philwhln
Blog : http://www.philwhln.com
Skype : philwhelan76