Return-Path: 
 <mapreduce-issues-return-54170-apmail-hadoop-mapreduce-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id C83D3D838
	for <apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org>;
 Tue, 13 Nov 2012 16:16:13 +0000 (UTC)
Received: (qmail 39130 invoked by uid 500); 13 Nov 2012 16:16:13 -0000
Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org
Received: (qmail 38858 invoked by uid 500); 13 Nov 2012 16:16:13 -0000
Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-issues@hadoop.apache.org>
List-Id: <mapreduce-issues.hadoop.apache.org>
Reply-To: mapreduce-issues@hadoop.apache.org
Delivered-To: mailing list mapreduce-issues@hadoop.apache.org
Received: (qmail 38811 invoked by uid 99); 13 Nov 2012 16:16:12 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Nov 2012 16:16:12 +0000
Date: Tue, 13 Nov 2012 16:16:12 +0000 (UTC)
From: "Thomas Graves (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <1826246710.108453.1352823372442.JavaMail.jiratomcat@arcas>
In-Reply-To: <567310599.15128.1344195722381.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (MAPREDUCE-4517) Too many INFO messages written
 out during AM to RM heartbeat
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/MAPREDUCE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496292#comment-13496292 ] 

Thomas Graves commented on MAPREDUCE-4517:
------------------------------------------

Mostly looks good. How about we take one more of the updateAndLogIfChanged calls out though and only call it once before Scheduling and then once after the assign (outside of the if block) and then in getResource add a log message for the # new or finished containers. This way we cut the logging down but I think still get the crucial information.  I think assign and getResource already have other log messages that will allow us to build up what happens in between.
                
> Too many INFO messages written out during AM to RM heartbeat
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-4517
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4517
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>    Affects Versions: 0.23.1, 2.0.1-alpha
>            Reporter: James Kinley
>            Assignee: Jason Lowe
>            Priority: Minor
>              Labels: patch
>         Attachments: MAPREDUCE-4517.patch, MAPREDUCE-4517.patch, MAPREDUCE-4517.patch, MAPREDUCE-4517.patch
>
>
> Too many INFO log messages written out during AM to RM heartbeat. Based on default frequency of 1000ms (scheduler.heartbeat.interval-ms) either 2 or 4 INFO messages are written out per second:
> LOG.info("Before Scheduling: " + getStat());
> List<Container> allocatedContainers = getResources();
> LOG.info("After Scheduling: " + getStat());
> if (allocatedContainers.size() > 0) {
>   LOG.info("Before Assign: " + getStat());
>   scheduledRequests.assign(allocatedContainers);
>   LOG.info("After Assign: " + getStat());
> }
> These should probably be changed to DEBUG message to save the log growing too quickly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira