hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maysam Yabandeh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-779) AMRMClient should clean up dangling unsatisfied request
Date Sat, 10 Aug 2013 09:14:50 GMT

     [ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Maysam Yabandeh updated YARN-779:
---------------------------------

    Attachment: YARN-779.patch

The attached patch includes a unit test that shows how the bug manifests in the discussed
erroneous scenario. If we are going to go for the suggested solution of changing API between
AM and RM, perhaps it would be better to open a new jira with a more descriptive title.
                
> AMRMClient should clean up dangling unsatisfied request
> -------------------------------------------------------
>
>                 Key: YARN-779
>                 URL: https://issues.apache.org/jira/browse/YARN-779
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 2.0.4-alpha
>            Reporter: Alejandro Abdelnur
>            Priority: Critical
>         Attachments: YARN-779.patch
>
>
> If an AMRMClient allocates a ContainerRequest for 10 containers in node1 or node2 is
placed (assuming a single rack) the resulting ResourceRequests will be
> {code}
> location - containers
> ---------------------
> node1    - 10
> node2    - 10
> rack     - 10
> ANY      - 10
> {code}
> Assuming 5 containers are allocated in node1 and 5 containers are allocated in node2,
the following ResourceRequests will be outstanding on the RM.
> {code}
> location - containers
> ---------------------
> node1    - 5
> node2    - 5
> {code}
> If the AMMRClient does a new ContainerRequest allocation, this time for 5 containers
in node3, the resulting outstanding ResourceRequests on the RM will be:
> {code}
> location - containers
> ---------------------
> node1    - 5
> node2    - 5
> node3    - 5
> rack     - 5
> ANY      - 5
> {code}
> At this point, the scheduler may assign 5 containers to node1 and it will never assign
the 5 containers node3 asked for.
> AMRMClient should keep track of the outstanding allocations counts per ContainerRequest
and when gets to zero it should update the the RACK/ANY decrementing the dangling requests.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message