hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: task slots problem
Date Tue, 25 Jan 2011 15:08:02 GMT
This is fine. 40 may be ending, while the next 40 may be starting. The tasks
will run a cleanup operation at their end (and be in such a 'status'),
during which the TaskTracker is allowed to schedule the next wave of maps it
needs. In these moments, it may appear as if 80 concurrent tasks may be
running. You can verify this by looking at the state of the 80 'running'
tasks. 40~ of them would be initializing while 40~ of them would be cleaning
up.

Reducer slots are not abused by the Mappers.

On Tue, Jan 25, 2011 at 6:33 PM, exception <exception@taomee.com> wrote:

>  Hi,
>
>
>
> My cluster contains 5 DataNodes, each with 8 map slots and 2 reduce slots.
> So there are up to 40 slots in my cluster and 40 tasks can run in parallel.
>
> But when running a particular job, I have noticed that 80 tasks running in
> parallel. The cluster looks fine when running other jobs.
>
>
> --
Harsh J
www.harshj.com

Mime
View raw message