Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 53084D6DC for ; Mon, 28 Jan 2013 03:03:01 +0000 (UTC) Received: (qmail 34641 invoked by uid 500); 28 Jan 2013 03:02:56 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 34413 invoked by uid 500); 28 Jan 2013 03:02:56 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 34394 invoked by uid 99); 28 Jan 2013 03:02:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Jan 2013 03:02:55 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of linlma@gmail.com designates 209.85.220.172 as permitted sender) Received: from [209.85.220.172] (HELO mail-vc0-f172.google.com) (209.85.220.172) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Jan 2013 03:02:49 +0000 Received: by mail-vc0-f172.google.com with SMTP id l6so1574151vcl.31 for ; Sun, 27 Jan 2013 19:02:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=55H8Kyv2Yk5rRcvXPw0cNxVN/dIGUJ45/qOv+YVJdZQ=; b=s0zBwXP4+mXs3n4eM9E+ZdkiGbTEZqjAJwVzBvb2kqXPSIweWAYDKYpgQaFl0RwqeK a6Enrfygd6n8AItkSm5yZK5/7JYWmkj85WZ9sUgu7wg23M17dmGUAZd9AVhMlGHSzgzL FguM7zptHJg5qw5WUyYdI4cfTShIOGDXnf0Yy90TIK2QRXqJN8XCuoT5X/IAxKv3Auhi 3mn9M4UwKyNoSWY77RdsBjhIlCFTXw/ziDHj2EcuSccVltW88DCu5zjbDqBJlvKL0i27 cS69Ta1P33wQ1jEVWHQZCzPktN0FWwQqqNUnShj5vocyCayXWrDzYl23kFcCCPEwr51K fCnQ== MIME-Version: 1.0 X-Received: by 10.52.89.48 with SMTP id bl16mr11724643vdb.120.1359342148354; Sun, 27 Jan 2013 19:02:28 -0800 (PST) Received: by 10.58.156.39 with HTTP; Sun, 27 Jan 2013 19:02:28 -0800 (PST) Date: Mon, 28 Jan 2013 11:02:28 +0800 Message-ID: Subject: hadoop queue From: Lin Ma To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=20cf307d03eae1849d04d45081a1 X-Virus-Checked: Checked by ClamAV on apache.org --20cf307d03eae1849d04d45081a1 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi guys, It is mentioned below for Hadoop queue ( http://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/), for the comments "FIFO scheduling with priorities is used", does it mean for resources (mapper/reducer slots) allocation inside each queue, two factors ((1) job submit order, i.e. FIFO, and (2) individual job priorities are also considered)? So it means even if inside one queue, the resource allocation is not fair between jobs? *However, within each queue, FIFO scheduling with priorities is used, except for one aspect =96 you can place a limit on percent of running tasks per user, so that users share a cluster equally. In other words, the capacity scheduler tries to simulate a separate FIFO/priority cluster for each user and each organization, rather than performing fair sharing between all jobs. * regards, Lin --20cf307d03eae1849d04d45081a1 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi guys,

It is mentioned below for Hadoop queue (http://blog.cloude= ra.com/blog/2008/11/job-scheduling-in-hadoop/), for the comments "= FIFO scheduling with priorities is used", does it mean for resources (= mapper/reducer slots) allocation inside each queue, two factors ((1) job su= bmit order, i.e. FIFO, and (2) individual job priorities are also considere= d)? So it means even if inside one queue, the resource allocation is not fa= ir between jobs?

However, within each queue, FIFO scheduling with priorities is used,= =20 except for one aspect =96 you can place a limit on percent of running=20 tasks per user, so that users share a cluster equally. In other words,=20 the capacity scheduler tries to simulate a separate FIFO/priority=20 cluster for each user and each organization, rather than performing fair sharing between all jobs.

regards,
Lin


--20cf307d03eae1849d04d45081a1--