hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Liochon <nkey...@gmail.com>
Subject Re: rack awarness unexpected behaviour
Date Thu, 22 Aug 2013 08:57:56 GMT
When you rebalance, the block is fully written, so the writer locality does
not have to be taken into account (there is no writer anymore), hence it
can rebalance across the racks. That's why jobs asymmetry was the easy
guess. What's your hadoop version by the way? I remember a bug around rack
awareness, but it has been fixed a year ago (and I'm not sure it would have
had this effect).

On Thu, Aug 22, 2013 at 10:29 AM, Marc Sturlese <marc.sturlese@gmail.com>wrote:

> Jobs run on the whole cluster. After rebalancing everything is properly
> allocated. Then I start running jobs using all the slots of the 2 racks and
> the problem starts to happen.
> Maybe I'm missing something. When using the rack awareness, do you have to
> specify to the jobs to run in slots form both racks and not just one? (I
> guess not)
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/rack-awareness-unexpected-behaviour-tp4086029p4086038.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message