hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M. C. Srivas" <mcsri...@gmail.com>
Subject Re: Hadoop cluster hardware details for big data
Date Wed, 06 Jul 2011 15:37:27 GMT
We ran the following on a 10+1 machine cluster (2-quad core, 24G DRAM,
12x2TB drives, 2 NICs each) running the 0.20.2  release

- 3.5TB terasort took ~4.5 hrs
- 10TB terasort  took ~12.5 hrs
- 20TB terasort  took > 24hrs

So yeah, Hadoop can handle it. If you want faster times, you'll have to try
- adding more machines
- using some other distro
- or both

On Wed, Jul 6, 2011 at 3:43 AM, Karthik Kumar <karthik84kumar@gmail.com>wrote:

> Hi,
>
> Has anyone here used hadoop to process more than 3TB of data? If so we
> would like to know how many machines you used in your cluster and
> about the hardware configuration. The objective is to know how to
> handle huge data in Hadoop cluster.
>
> --
> With Regards,
> Karthik
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message