hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13791) Limit logging frequency of edit tail related statements
Date Fri, 21 Sep 2018 21:52:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624230#comment-16624230

Erik Krogen commented on HDFS-13791:

Attached a v005 patch which adds in a decent unit test for the behavior, and also pulls out
some duplicated logic into a static helper on {{LogThrottlingHelper}} (constructing a message
about the number of suppressed statements).

> Limit logging frequency of edit tail related statements
> -------------------------------------------------------
>                 Key: HDFS-13791
>                 URL: https://issues.apache.org/jira/browse/HDFS-13791
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs, qjm
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>            Priority: Major
>         Attachments: HDFS-13791-HDFS-12943.000.patch, HDFS-13791-HDFS-12943.001.patch,
HDFS-13791-HDFS-12943.002.patch, HDFS-13791-HDFS-12943.003.patch, HDFS-13791-HDFS-12943.004.patch,
> There are a number of log statements that occur every time new edits are tailed by a
Standby NameNode. When edits are tailing only on the order of every tens of seconds, this
is fine. With the work in HDFS-13150, however, edits may be tailed every few milliseconds,
which can flood the logs with tailing-related statements. We should throttle it to limit it
to printing at most, say, once per 5 seconds.
> We can implement logic similar to that used in HDFS-10713. This may be slightly more
tricky since the log statements are distributed across a few classes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message