hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen mulcahy <stephen.mulc...@deri.org>
Subject Re: Bisection bandwidth needed for a small Hadoop cluster
Date Wed, 27 May 2009 17:51:42 GMT
Todd Lipcon wrote:
> Hi Stephen,
> 
> The true answer depends on the types of jobs you're running. As a back of
> the envelope calculation I might figure something like this:
> ...
> So, the simple answer is that 4-6Gbps is most likely just fine for most
> practical jobs. If you want to be extra safe, many inexpensive switches can
> operate in a "stacked" configuration where the bandwidth between them is
> essentially backplane speed. That should scale you to 96 nodes with plenty
> of headroom.

Hi Todd,

Many thanks for the detailed response - I understand it all depends on 
my specific job and that the usual YMMV caveat applies but your figures 
are very helpful to give me some ballpark figures to play with.

 From what I can see, the inexpensive switches use add-on modules in 
stacking configurations and those add-on modules are reasonably pricey 
(especially if you want to stack switches with 10 Gbps links rather than 
1 Gbps links) so the ballpark number is pretty useful from a budgeting 
perspective.

Thanks,

-stephen

-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Mime
View raw message