hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wilfred Spiegelenburg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level
Date Mon, 16 Mar 2015 05:43:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362771#comment-14362771

Wilfred Spiegelenburg commented on YARN-3350:

No tests for this change: it is just a simple log level change without any further code changes.

BTW: this same issue was encountered by Spark as an application using the RackResolver and
they changed the log level from their side to prevent log flooding via SPARK-5393

> YARN RackResolver spams logs with messages at info level
> --------------------------------------------------------
>                 Key: YARN-3350
>                 URL: https://issues.apache.org/jira/browse/YARN-3350
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.6.0
>            Reporter: Wilfred Spiegelenburg
>            Assignee: Wilfred Spiegelenburg
>         Attachments: YARN-3350.patch, yarn-RackResolver-log.txt
> When you run an application the container logs shows a lot of messages for the RackResolver:
> 2015-03-10 00:58:30,483 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver:
Resolved node175.example.com to /rack15
> A real world example for a large job was generating 20+ messages in 2 milliseconds during
a sustained period of time flooding the logs causing the node to run out of disk space.

This message was sent by Atlassian JIRA

View raw message