hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject RE: Taking advantage of multiple networks
Date Fri, 20 Aug 2010 22:45:52 GMT



> From: alexey@kovyrin.net
> Date: Fri, 20 Aug 2010 11:48:51 -0400
> Subject: Re: Taking advantage of multiple networks
> To: user@hbase.apache.org
> 
> Not sure about hadoop support, but OS-level (or maybe even network
> card level) port bonding looks like the way to go.
> 

Yes, port bonding is possible and should be 'invisible' to HBase.

Do you have Ganglia installed and if so, what do you see with respect to network traffic when
your large jobs are running?

It could be that the network isn't the bottleneck but a contributing factor.

-Mike

> On Fri, Aug 20, 2010 at 11:36 AM, Buttler, David <buttler1@llnl.gov> wrote:
> > Hi all,
> > I have a small hbase cluster that I have recently filled with about 500M records
(some of them quite large).  One of the things that I notice when I do different types of
map / reduce jobs over my table is that the network becomes a bottleneck.  Currently I am
running single gig Ethernet on this cluster, but it has 4 network ports.
> >
> > My question is this: is it possible to set up hadoop/hbase to take advantage of
multiple networks connecting the computers?
> > Could I specify multiple network connections in the config file?
> > Would it make sense to put the region servers on a different network than the data
nodes?
> > Would it be more efficient to bond multiple channels at the OS level?
> 
> -- 
> Alexey Kovyrin
> http://kovyrin.net/
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message