hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GOEKE, MATTHEW (AG/1000)" <matthew.go...@monsanto.com>
Subject RE: hadoop cluster on VM's
Date Mon, 15 Aug 2011 20:15:38 GMT
I was referring to multiple VM's on a single machine (that you have in house) for my previous
comment and not EC2. FWIW, I would rather see a single heavy data node than to partition off
a single box into multiple machines unless you are trying to do more on that server than just
Hadoop. Obviously every person / company has their own constraints but if this box is solely
for Hadoop then don't partition it otherwise you will incur a decent loss in possible map/reduce


-----Original Message-----
From: Liam Friel [mailto:liam.friel@gmail.com] 
Sent: Monday, August 15, 2011 3:04 PM
To: common-user@hadoop.apache.org
Subject: Re: hadoop cluster on VM's

On Mon, Aug 15, 2011 at 7:31 PM, GOEKE, MATTHEW (AG/1000) <
matthew.goeke@monsanto.com> wrote:

> Is this just for testing purposes or are you planning on going into
> production with this? If it is the latter than I would STRONGLY advise to
> not give that a second thought due to how the framework handles I/O. However
> if you are just trying to test out distributed daemon setup and get some ops
> documentation then have at it :)
> Matt
> -----Original Message-----
> From: Travis Camechis [mailto:camechis@gmail.com]
> Sent: Monday, August 15, 2011 12:45 PM
> To: common-user@hadoop.apache.org
> Subject: hadoop cluster on VM's
> Is it recommended to install a hadoop cluster on a set of VM's that are all
> connected to a SAN?
Could you expand on that? Do you mean multiple VMs on a single server are a
Or do you mean running Hadoop on something like Amazon EC2 for production is
also a no-no?
With some pointers to background if the latter please ...

Just for my education. I have run some (test I guess you could call them)
Hadoop clusters on EC2 and it was working OK.
However I didn't have the equivalent pile of physical hardware lying around
to do a comparison ... which I guess is why EC2 is so attractive.

This e-mail message may contain privileged and/or confidential information, and is intended
to be received only by persons entitled
to receive such information. If you have received this e-mail in error, please notify the
sender immediately. Please delete it and
all attachments from any servers, hard drives or any other media. Other use of this e-mail
by you is strictly prohibited.

All e-mails and attachments sent and received are subject to monitoring, reading and archival
by Monsanto, including its
subsidiaries. The recipient of this e-mail is solely responsible for checking for the presence
of "Viruses" or other "Malware".
Monsanto, along with its subsidiaries, accepts no liability for any damage caused by any such
code transmitted by or accompanying
this e-mail or any attachment.

The information contained in this email may be subject to the export control laws and regulations
of the United States, potentially
including but not limited to the Export Administration Regulations (EAR) and sanctions regulations
issued by the U.S. Department of
Treasury, Office of Foreign Asset Controls (OFAC).  As a recipient of this information you
are obligated to comply with all
applicable U.S. export laws and regulations.

View raw message