hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: Hadoop Cluster Survey
Date Mon, 13 Jul 2009 16:32:20 GMT
On Jul 12, 2009, at 12:21 PM, Jon Miller wrote:

> I realize what I'm asking is highly subjective to the hardware, number
> of nodes, how large the data set is, etc and not to mention the
> particular calculation being performed. Realizing this, I know it's
> nearly impossible to provide a prediction to the readers but perhaps
> if I can survey this mailing list with a few questions, then maybe I
> can develop a nice heuristic formula which can be used?

I'd start with the Hadoop Powered By page:
http://wiki.apache.org/hadoop/PoweredBy

Of course the numbers run 3-6 months stale, but they are a good first  
pass.

I'd also suggest you look through my petabyte sort blog entry:

http://developer.yahoo.net/blogs/hadoop/2009/05/hadoop_sorts_a_petabyte_in_162.html

I'd also suggest you watch the opening talk of the Hadoop Summit '09  
where Eric, and I cover a lot of this stuff with respect to Yahoo:

http://developer.yahoo.net/blogs/theater/archives/2009/06/hadoopsummit_omalley.html

-- Owen

Mime
View raw message