hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Ravi <bharathra...@gmail.com>
Subject Re: Load balancing requests in HDFS
Date Sun, 16 Oct 2011 19:11:47 GMT
Thanks a lot Harsh!
I'll look into this jira issue!

On 16 October 2011 08:15, Harsh J <harsh@cloudera.com> wrote:

> Bharath,
>
> You may want to take a look at
> http://issues.apache.org/jira/browse/HDFS-782 and continue the
> discussion on that.
>
> On Sun, Oct 16, 2011 at 7:23 AM, Bharath Ravi <bharathravi1@gmail.com>
> wrote:
> > Hi all,
> >
> > I have a question about how HDFS load balances requests for files/blocks:
> >
> > HDFS currently distributes data blocks randomly, for balance.
> > However, if certain files/blocks are more popular than others, some nodes
> > might get an "unfair" number of requests.
> > Adding more replicas for these popular files might not help, unless HDFS
> > explicitly distributes requests fairly among the replicas.
> >
> > Looking around, I saw a bit of discussion on this dating to 2009,
> > Is something like this is done by HDFS today?
> >
> > http://www.mail-archive.com/core-user@hadoop.apache.org/msg04357.html
> >
> http://grokbase.com/p/hadoop.apache.org/common-dev/2009/02/re-hdfs-load-traffic-balancing/18en5o4z2j4b6eaqnemkp4eppd7q
> >
> > Thanks a lot in advance!
> > --
> > Bharath Ravi
> >
>
>
>
> --
> Harsh J
>



-- 
Bharath Ravi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message