hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: What's HDFS's balancer criteria?
Date Fri, 18 Dec 2009 17:03:32 GMT
Hi Jeff,

You can tune dfs.datanode.balance.bandwidthPerSec in order to change the
speed at which the balancer moves data around. It defaults to 1MB/sec so as
to avoid using significant cluster resources, but you could certainly bump
it up for an individual balancer run if you need it to go quicker.

-Todd

On Fri, Dec 18, 2009 at 5:05 AM, Jeff Zhang <zjffdu@gmail.com> wrote:

> It's my mistake, the balancer process is so slow that make me think it
> balance regarding the block
>
> Jeff Zhang
>
> On Fri, Dec 18, 2009 at 8:36 PM, Brian Bockelman <bbockelm@cse.unl.edu
> >wrote:
>
> > Hey Jeff,
> >
> > I have not encountered this.
> >
> > http://dcache-head.unl.edu:8088/dfshealth.jsp
> >
> > The nodes that are 10x bigger have 10x more blocks on them.  What makes
> you
> > think this is true?
> >
> > Brian
> >
> > On Dec 18, 2009, at 4:29 AM, Jeff Zhang wrote:
> >
> > > Hi all,
> > >
> > > When I invoke the balancer command, I found that it seems hdfs want to
> > > balance the block number of each data node rather than the percentage
> of
> > > each machine's remaining capacity.
> > > Not sure if I am correct, could any help on this ?
> > >
> > >
> > > Thank you
> > >
> > > Jeff Zhang
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message