hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4390) Consider container request size during CS preemption
Date Tue, 24 Nov 2015 22:13:10 GMT

    [ https://issues.apache.org/jira/browse/YARN-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025542#comment-15025542
] 

Bikas Saha commented on YARN-4390:
----------------------------------

I am not sure if this is a bug as described. If preemption does free 8x1GB containers then
it will create 8GB free space on the node. The scheduler (which is container request size
aware) should then allocate 1x8GB container to the under-allocated AM. [~curino] Is that correct?
Of course there could be a bug in the implementation but by design, this should not happening.

However, if YARN ends up preempting 8x1GB containers on different nodes then the under-allocated
AM will not get its resources and may result in further avoidable preemptions. This is [~sunilg]
case.

> Consider container request size during CS preemption
> ----------------------------------------------------
>
>                 Key: YARN-4390
>                 URL: https://issues.apache.org/jira/browse/YARN-4390
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>    Affects Versions: 3.0.0, 2.8.0, 2.7.3
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>
> There are multiple reasons why preemption could unnecessarily preempt containers. One
is that an app could be requesting a large container (say 8-GB), and the preemption monitor
could conceivably preempt multiple containers (say 8, 1-GB containers) in order to fill the
large container request. These smaller containers would then be rejected by the requesting
AM and potentially given right back to the preempted app.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message