Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: Evert Lammerts <Evert.Lammerts@sara.nl>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Subject: RE: Pending reducers
Thread-Topic: Pending reducers
Thread-Index: Ac16CP70q1VWsSrCRNONZJm3+wtl+v//4SIA///NqbA=
Date: Tue, 14 Aug 2012 11:50:12 +0000
Deferred-Delivery: Tue, 14 Aug 2012 11:52:00 +0000
Message-ID: <7AFB227D47B00A49A4C9E00FF82685B42E90E4@sara-exch-2.ka.sara.nl>
References: <7AFB227D47B00A49A4C9E00FF82685B42E8E01@sara-exch-2.ka.sara.nl>
 <CAORpBsj87=KvTNkGp0EB1XuCWY-s1o5xB==uiVL_RGRZr7nOJw@mail.gmail.com>
In-Reply-To: 
 <CAORpBsj87=KvTNkGp0EB1XuCWY-s1o5xB==uiVL_RGRZr7nOJw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

> reducers of multiple jobs do run con-currently as long as they have the
> resources available.

Yep, and that's what's not happening in my situation. 528 reduce slots, 400=
 taken by one job, 26 of another job remain in pending state. What could ex=
plain this behavior?

Evert

>=20
> If you want to limit someone overtaking the cluster, then you can
> create different job queues and assign quota to each queue. You also
> have the flexibility of allocating max quota per user in a queue as
> well.
>=20
>=20
>=20
> On Tue, Aug 14, 2012 at 4:09 PM, Evert Lammerts
> <Evert.Lammerts@sara.nl> wrote:
> > Hi list,
> >
> > I have a cluster running Hadoop 0.20.205 with Kerberos enabled,
> exposing 528 map slots and 528 reduce slots. Currently somebody is
> running a NORMAL priority job with 7 mappers and 400 reducers. The
> mappers have finished and the system is processing the reducers.
> Another user is running a NORMAL priority job with 1 mapper and 26
> reducers. The mapper has finished, but the reducers won't come out of
> "pending" state. There are no other jobs running right now. We've not
> yet installed a different scheduler, so right now the system is using
> the default scheduler. How can this behavior be explained? I see
> mappers of multiple jobs run concurrently, and I *thought* I've seen
> reducers of multiple jobs run concurrently, but I'm not completely
> sure. Any idea?
> >
> > Evert
>=20
>=20
>=20
> --
> Nitin Pawar