hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John George (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user
Date Mon, 03 Oct 2011 22:32:35 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

John George updated MAPREDUCE-2794:

    Attachment: MAPREDUCE-2794.patch

There are two issues mentioned in this JIRA:
1. AvailableGB metrics for user queue is higher than what it is for the default queue

2. AvailableGB does not reflect the "real time" AvailableGB in the case of user queue.

Issue (1) is because user queue rounds up to the next "minimum allocation". This patch tries
to round up all the leaf queues (not just the user leaf queue) to the next minimum allocation.

Issue (2) is because AvailableGB is not updated every time an allocation is made.

The uploaded patch is to check if the approach is acceptable or if there are any other suggestions
to solve this...
> [MR-279] Incorrect metrics value for AvailableGB per queue per user
> -------------------------------------------------------------------
>                 Key: MAPREDUCE-2794
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: John George
>            Priority: Blocker
>             Fix For: 0.23.0
>         Attachments: MAPREDUCE-2794.patch
> AvailableGB per queue is not the same as AvailableGB per queue per user when the user
limit is set to 100%.
> i.e. if the total available GB of the cluster is 60, and queue "default" has 92% capacity
with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB
per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) 
> Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user
ramya is running apps on the "default" queue.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message