Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6945211EC2 for ; Wed, 16 Jul 2014 08:30:06 +0000 (UTC) Received: (qmail 75056 invoked by uid 500); 16 Jul 2014 08:30:06 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 75001 invoked by uid 500); 16 Jul 2014 08:30:06 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 74975 invoked by uid 99); 16 Jul 2014 08:30:06 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Jul 2014 08:30:06 +0000 Date: Wed, 16 Jul 2014 08:30:05 +0000 (UTC) From: "Chris Douglas (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (YARN-2297) Preemption can hang when configured ridiculously MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated YARN-2297: -------------------------------- Summary: Preemption can hang when configured ridiculously (was: Preemption can hang in corner case by not allowing any task container to proceed.) > Preemption can hang when configured ridiculously > ------------------------------------------------ > > Key: YARN-2297 > URL: https://issues.apache.org/jira/browse/YARN-2297 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler > Affects Versions: 2.5.0 > Reporter: Tassapol Athiapinya > Assignee: Wangda Tan > Priority: Critical > > Preemption can cause hang issue in single-node cluster. Only AMs run. No task container can run. > h3. queue configuration > Queue A/B has 1% and 99% respectively. > No max capacity. > h3. scenario > Turn on preemption. Configure 1 NM with 4 GB of memory. Use only 2 apps. Use 1 user. > Submit app 1 to queue A. AM needs 2 GB. There is 1 task that needs 2 GB. Occupy entire cluster. > Submit app 2 to queue B. AM needs 2 GB. There are 3 tasks that need 2 GB each. > Instead of entire app 1 preempted, app 1 AM will stay. App 2 AM will launch. No task of either app can proceed. > h3. commands > /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar randomtextwriter "-Dmapreduce.map.memory.mb=2000" "-Dyarn.app.mapreduce.am.command-opts=-Xmx1800M" "-Dmapreduce.randomtextwriter.bytespermap=2147483648" "-Dmapreduce.job.queuename=A" "-Dmapreduce.map.maxattempts=100" "-Dmapreduce.am.max-attempts=1" "-Dyarn.app.mapreduce.am.resource.mb=2000" "-Dmapreduce.map.java.opts=-Xmx1800M" "-Dmapreduce.randomtextwriter.mapsperhost=1" "-Dmapreduce.randomtextwriter.totalbytes=2147483648" dir1 > /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-tests.jar sleep "-Dmapreduce.map.memory.mb=2000" "-Dyarn.app.mapreduce.am.command-opts=-Xmx1800M" "-Dmapreduce.job.queuename=B" "-Dmapreduce.map.maxattempts=100" "-Dmapreduce.am.max-attempts=1" "-Dyarn.app.mapreduce.am.resource.mb=2000" "-Dmapreduce.map.java.opts=-Xmx1800M" -m 1 -r 0 -mt 4000 -rt 0 -- This message was sent by Atlassian JIRA (v6.2#6252)