hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: Thoughts about Hadoop cluster hardware
Date Wed, 14 Jul 2010 03:36:21 GMT

On Jul 13, 2010, at 5:00 PM, u235sentinel wrote:

> So we're talking to Dell about their new PowerEdge c2100 servers for a Hadoop cluster
but I'm wondering.  Isn't this still a little overboard for nodes in a cluster?  I'm wondering
if we bought say 100 poweredge 2750's instead of just 50 c2100's.  The price would be about
the same for the configuration we're talking about and we would get twice as many nodes.

Ultimately, it depends upon your job flow and how much data you have.  

FWIW we're currently using a Sun equivalent of the C2100s w/8 of the 12 drive slots filled.
 You need a *LOT* of iops to make it worth while.  [From what I've seen, even people who think
they have a lot of iops generally have other problems with their code/tuning that are causing
the iops.   So even if you think you have a lot, you may not.]

> I'm curious if any other's are running Dell PowerEdge servers with Hadoop.
> 
> We've also been kicking the idea around of going with blade servers (Dell and/or HP).

If you are thinking traditional blade where storage is comes mainly from NAS or SAN, you are
going to be very, very unhappy unless your data set is very, very tiny.

Check out the PoweredBy page on the wiki.  Quite a few folks list their gear. FWIW, we're
currently evaluating HP SLs and should be getting some Dell C6100s in soon, assuming Dell
can deliver the eval unit on time.
Mime
View raw message