hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <markkerz...@gmail.com>
Subject Re: How hard would it be to put Hadoop on an EC2 machine?
Date Mon, 16 Nov 2009 13:28:09 GMT
Tim, thank you,

these are important considerations, making one lean toward using prepared
EC2 Hadoop distributions.

Mark

On Mon, Nov 16, 2009 at 12:20 AM, Tim Robertson
<timrobertson100@gmail.com>wrote:

> Hi Mark,
>
> I have never done it, but I *think* it would be reasonably easy.  You
> will have to manipulate a few scripts which you'd run each time you
> start up a cluster, to copy around the addresses of the EC2 machines
> (e.g. copy the master and slaves files etc).  That would be a fairly
> simple admin task each time you start.
> You will also need to do the passphraseless ssh between the machines,
> which you should be able to script quite easily also.
>
> Other than that, I think it would be just copying a directory into
> your AMI and manipulating the config as per any hadoop install (the 3
> *-site.xml config files).
>
> There are nice scripts for EC2 with Hadoop, and Cloudera have done a
> lot of investment here as well... might be worth considering how easy
> it is to port your stuff to their AMI (HBase, Hive, Cascading etc all
> configured as well), and also bear in mind how quickly Hadoop is
> evolving so there will be ongoing maintenance.
>
> Cheers,
> Tim
>
>
>
>
> On Mon, Nov 16, 2009 at 6:34 AM, Jeff Zhang <zjffdu@gmail.com> wrote:
> > You can use the distribution of cloudera, it's very easy to install
> hadoop
> > on ec2.
> >
> > http://archive.cloudera.com/docs/ec2.html
> >
> >
> > Jeff Zhang
> >
> >
> > On Mon, Nov 16, 2009 at 1:24 PM, Mark Kerzner <markkerzner@gmail.com>
> wrote:
> >
> >> Hi,
> >>
> >> ideally I would like to take the EC2 image that suits my purpose, then
> add
> >> Hadoop to it, save this image and use it for my cluster. How hard would
> it
> >> be? What additional configuration do I need to do after I install hadoop
> on
> >> the machine?
> >>
> >> Thank you,
> >> Mark
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message