Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 35195 invoked from network); 13 Dec 2008 20:03:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 13 Dec 2008 20:03:11 -0000 Received: (qmail 17111 invoked by uid 500); 13 Dec 2008 20:03:19 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 16558 invoked by uid 500); 13 Dec 2008 20:03:17 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 16535 invoked by uid 99); 13 Dec 2008 20:03:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Dec 2008 12:03:17 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Dec 2008 20:03:04 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 76F74234C3D0 for ; Sat, 13 Dec 2008 12:02:44 -0800 (PST) Message-ID: <2043717711.1229198564486.JavaMail.jira@brutus> Date: Sat, 13 Dec 2008 12:02:44 -0800 (PST) From: "Matei Zaharia (JIRA)" To: core-dev@hadoop.apache.org Subject: [jira] Updated: (HADOOP-4789) Change fair scheduler to share between pools by default, not between invidual jobs In-Reply-To: <409924871.1228509404275.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated HADOOP-4789: ---------------------------------- Status: Patch Available (was: Open) > Change fair scheduler to share between pools by default, not between invidual jobs > ---------------------------------------------------------------------------------- > > Key: HADOOP-4789 > URL: https://issues.apache.org/jira/browse/HADOOP-4789 > Project: Hadoop Core > Issue Type: Improvement > Components: contrib/fair-share > Reporter: Matei Zaharia > Assignee: Matei Zaharia > Priority: Minor > Attachments: hadoop-4789-v1.patch > > > The fair scheduler currently treats jobs as equal entities in sharing by default, so that a user who submits 2 jobs gets 2x the share of a user who submits only 1 job. We found that it makes more sense to support equal shares between individual pools instead, and have one pool per user, because users can otherwise game the system by submitting multiple small jobs. This patch will set the scheduler to share between pools by default and set the default pool assignment process to one pool per user. it will also be possible to give weights to pools so that some users/groups/applications get a larger share of the cluster if they really do need to run more jobs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.