Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Date: Sat, 16 Jul 2011 22:13:59 +0000 (UTC)
From: "Arun C Murthy (JIRA)" <jira@apache.org>
To: mapreduce-dev@hadoop.apache.org
Message-ID: 
 <256722318.20434.1310854439914.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1353413269.19066.1310769600008.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Resolved] (MAPREDUCE-2694) AM releases too many containers
 due to the protocol
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun C Murthy resolved MAPREDUCE-2694.
--------------------------------------

    Resolution: Not A Problem

I thought more about this... I've spent time on this in the beginning and I don't think a delta protocol is appropriate - particularly since the RM and AM aren't 'in sync' ever.

Yes, this could lead to some waste, but the system is eventually consistent.

> AM releases too many containers due to the protocol
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-2694
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2694
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>
> - AM sends request asking 4 containers on host H1.
> - Asynchronously, host H1 reaches RM and gets assigned 4 containers. RM at this point, sets the value against H1 to
> zero in its aggregate request-table for all apps.
> - In the mean-while AM gets to need 3 more containers, so a total of 7 including the 4 from previous request.
> - Today, AM sends the absolute number of 7 against H1 to RM as part of its request table.
> - RM seems to be overriding its earlier value of zero against H1 to 7 against H1. And thus allocating 7 more
> containers.
> - AM already gets 4 in this scheduling iteration, but gets 7 more, a total of 11 instead of the required 7.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira