hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4606) CapacityScheduler: applications could get starved because #activeUsers considers pending apps
Date Wed, 20 Jan 2016 07:39:39 GMT

     [ https://issues.apache.org/jira/browse/YARN-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wangda Tan updated YARN-4606:
-----------------------------
    Summary: CapacityScheduler: applications could get starved because #activeUsers considers
pending apps  (was: Sometimes Fairness inconjuncttions with UserLimitPercent and UserLimitFactor
in queue leads to situation where it appears that applications in queue are getting starved
or stuck)

> CapacityScheduler: applications could get starved because #activeUsers considers pending
apps
> ---------------------------------------------------------------------------------------------
>
>                 Key: YARN-4606
>                 URL: https://issues.apache.org/jira/browse/YARN-4606
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, capacityscheduler
>    Affects Versions: 2.8.0, 2.7.1
>            Reporter: Karam Singh
>            Assignee: Wangda Tan
>
> Encountered while studying behaviour fairness with UserLimitPercent and UserLimitFactor
during following test:
> Ran GridMix with Queue settings: Capacity=10, MaxCap=80, UserLimit=25 UserLimitFactor=32,
FairOrderingPolicy only. Encountered a application starving situation where 33 application
(190 apps completed out of 761 apps, queue can 345 containers) are running with total of 45
containers running, and that 12 extra only one app(the app was having around 18000 tasks)
, all other apps were having AM running only no other containers were given any apps. After
that app finished, there were 32 AMs that kept running without any containers for task being
launched
> GridMix was run with following settings:
> gridmix.client.pending.queue.depth=10, gridmix.job-submission.policy=REPLAY, gridmix.client.submit.threads=5,
gridmix.submit.multiplier=0.0001, gridmix.job.type=SLEEPJOB, mapreduce.framework.name=yarn,
mapreduce.job.queuename=hive1, mapred.job.queue.name=hive1, gridmix.sleep.max-map-time=5000,
gridmix.sleep.max-reduce-time=5000, gridmix.user.resolve.class=org.apache.hadoop.mapred.gridmix.RoundRobinUserResolver
With Users file containing 4 users for RoundRobinUserResolver



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message