hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <ja...@attributor.com>
Subject Re: Question on DFS block placement and 'what is a rack' wrt DFS block placement
Date Tue, 12 Feb 2008 23:01:21 GMT
Okay back to network topology then.

How does hadoop determine a 'rack' of machines?

Currently we have everything on a single VLAN, with the DFS master being 
the gateway back to our main VLAN.

We were hoping to group subsets of machines on a local switch, and 
optionally have each machine have a connection to the vlan which is the 
backbone of the entire cluster.

In terms of more exotic situations we were discussing having 4 NIC's 1 
for the local subset, 1 for a pair of local subsets, 1 for another pair 
of local subsets, 1 for the backbone.

Our job mix is totally varied from IO bound to CPU bound
Jason Venner
Attributor - Publish with Confidence <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers, contact if interested

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message