apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vlad Rozov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-777) Application Master may not shutdown due to incorrect numRequestedContainers counting
Date Tue, 15 Aug 2017 18:20:00 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127666#comment-16127666

Vlad Rozov commented on APEXCORE-777:

-1 means "must not". While I am +1 on code refactoring and agree that it needs to be done
to help with unit testing and readability/maintainability of the code, this particular issue
must be fixed as part of this JIRA and a separate PR.

> Application Master may not shutdown due to incorrect numRequestedContainers counting
> ------------------------------------------------------------------------------------
>                 Key: APEXCORE-777
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-777
>             Project: Apache Apex Core
>          Issue Type: Bug
>            Reporter: Vlad Rozov
>            Priority: Minor
> Consider a scenario where App master requests a container from Yarn (numRequestedContainers
= 1). There is not enough resources and the request timeouts. My understanding is that App
master will re-request it again but the number of requested containers will not change (one
newly requested, one removed). Let's assume that App master, by the time Yarn responds back
decides that it does not need any. If Yarn responds with one allocated containers, numRequestedContainers
will go to 0 (correct), but Yarn may respond back with 2 allocated containers if by the time
App Master sends the second request it already allocated a container requested in the original
request (the one that timeouted) as Yarn does not guarantee that removed request is fullfilled
(see Yarn doc). Will not in this case numRequestedContainers be -1 due to the bulk decrement?

This message was sent by Atlassian JIRA

View raw message