hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4517) Too many INFO messages written out during AM to RM heartbeat
Date Mon, 22 Oct 2012 15:52:13 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe updated MAPREDUCE-4517:

    Status: Open  (was: Patch Available)

I agree the messages are too long and printing them every heartbeat is wasteful.  However
the messages have proven multiple times to be critical in debugging scheduling issues, and
I fear having them turned off completely by default will make debugging issues between the
RM and AM very difficult.

I think we could log a lot less and still have the state info we need if we only logged these
messages when something in them changes.  Logging the same state every heartbeat isn't useful,
but knowing when these states change is.
> Too many INFO messages written out during AM to RM heartbeat
> ------------------------------------------------------------
>                 Key: MAPREDUCE-4517
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4517
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>            Reporter: James Kinley
>            Priority: Minor
>              Labels: patch
>             Fix For: trunk
>         Attachments: MAPREDUCE-4517.patch
> Too many INFO log messages written out during AM to RM heartbeat. Based on default frequency
of 1000ms (scheduler.heartbeat.interval-ms) either 2 or 4 INFO messages are written out per
> LOG.info("Before Scheduling: " + getStat());
> List<Container> allocatedContainers = getResources();
> LOG.info("After Scheduling: " + getStat());
> if (allocatedContainers.size() > 0) {
>   LOG.info("Before Assign: " + getStat());
>   scheduledRequests.assign(allocatedContainers);
>   LOG.info("After Assign: " + getStat());
> }
> These should probably be changed to DEBUG message to save the log growing too quickly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message