hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4517) Too many INFO messages written out during AM to RM heartbeat
Date Tue, 13 Nov 2012 16:16:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496292#comment-13496292
] 

Thomas Graves commented on MAPREDUCE-4517:
------------------------------------------

Mostly looks good. How about we take one more of the updateAndLogIfChanged calls out though
and only call it once before Scheduling and then once after the assign (outside of the if
block) and then in getResource add a log message for the # new or finished containers. This
way we cut the logging down but I think still get the crucial information.  I think assign
and getResource already have other log messages that will allow us to build up what happens
in between.
                
> Too many INFO messages written out during AM to RM heartbeat
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-4517
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4517
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>    Affects Versions: 0.23.1, 2.0.1-alpha
>            Reporter: James Kinley
>            Assignee: Jason Lowe
>            Priority: Minor
>              Labels: patch
>         Attachments: MAPREDUCE-4517.patch, MAPREDUCE-4517.patch, MAPREDUCE-4517.patch,
MAPREDUCE-4517.patch
>
>
> Too many INFO log messages written out during AM to RM heartbeat. Based on default frequency
of 1000ms (scheduler.heartbeat.interval-ms) either 2 or 4 INFO messages are written out per
second:
> LOG.info("Before Scheduling: " + getStat());
> List<Container> allocatedContainers = getResources();
> LOG.info("After Scheduling: " + getStat());
> if (allocatedContainers.size() > 0) {
>   LOG.info("Before Assign: " + getStat());
>   scheduledRequests.assign(allocatedContainers);
>   LOG.info("After Assign: " + getStat());
> }
> These should probably be changed to DEBUG message to save the log growing too quickly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message