hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4610) Reservations continue looking for one app causes other apps to starve
Date Wed, 20 Jan 2016 16:57:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108902#comment-15108902
] 

Jason Lowe commented on YARN-4610:
----------------------------------

bq. could you take a look at failed tests? Not sure if they're related to this fix.

The TestApplicationPriority failure appears to be unrelated.  I tried multiple times to reproduce
it with the patch applied, and it always passes.

The TestClientRMTokens failure is unrelated.  It's failing the same way for other precommit
builds, and is being tracked by YARN-4306 / HADOOP-12687.

The TestAMAuthorization failure is unrelated.  It's failing the same way for other precommit
builds, and is being tracked by YARN-4318 / HADOOP-12687.

The TestClientRMService failure is unrelated.  It's failing the same way in some other precommit
builds (e.g.: see https://builds.apache.org/job/PreCommit-YARN-Build/10285/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt).
 I tried running it multiple times with the patch applied, and it always passed.  I couldn't
find a tracking ticket for this one, so I filed YARN-4613.

I'm guessing the TestAbstractYarnScheduler is unrelated.  Note that it didn't fail with JDK7
but does with JDK8.  I tried isolating the test with JDK8, and it doesn't fail, yet it fails
when run as part of the suite.  I'll investigate it further to see if I can narrow down why
it's failing in JDK8, but the fact that it's passing on another JDK and also passes when run
in isolation indicates it's likely not a problem with the patch.

> Reservations continue looking for one app causes other apps to starve
> ---------------------------------------------------------------------
>
>                 Key: YARN-4610
>                 URL: https://issues.apache.org/jira/browse/YARN-4610
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.7.1
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Blocker
>         Attachments: YARN-4610.001.patch
>
>
> CapacityScheduler's LeafQueue has "reservations continue looking" logic that allows an
application to unreserve elsewhere to fulfil a container request on a node that has available
space.  However in 2.7 that logic seems to break allocations for subsequent apps in the queue.
 Once a user hits its user limit, subsequent apps in the queue for other users receive containers
at a significantly reduced rate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message