hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Angeles <patrickange...@gmail.com>
Subject Re: Advice on new Datacenter Hadoop Cluster?
Date Wed, 30 Sep 2009 14:33:53 GMT
We went with 2 x Nehalems, 4 x 1TB drives and 24GB RAM. The ram might be
overkill... but it's DDR3 so you get either 12 or 24GB. Each box has 16
virtual cores so 12GB might not have been enough. These boxes are around $4k
each, but can easily outperform any $1K box dollar per dollar (and
performance per watt).

If you're extremely I/O bound, you can get single-socket configurations with
the same amount of drive spindles for really cheap (~$2k for single proc,
8-12GB RAM, 4x1TB drives).

On Wed, Sep 30, 2009 at 10:19 AM, stephen mulcahy
<stephen.mulcahy@deri.org>wrote:

> Todd Lipcon wrote:
>
>> Most people building new clusters at this point seem to be leaning towards
>> dual quad core Nehalem with 4x1TB 7200RPM SATA and at least 8G RAM.
>>
>
> We went with a similar configuration for a recently purchased cluster but
> opted for qual quad core Opterons (Shanghai) rather than Nehalems and
> invested the difference in more memory per node (16GB). Nehalem seem to
> perform very well on some benchmarks but that performance comes at a
> premium. I guess it depends on your planned use of the cluster but in a lot
> of cases more memory may be better spent, especially if you plan on running
> things like HBase on the cluster also (which we do).
>
> -stephen
>
> --
> Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
> NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
> http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message