From common-issues-return-158198-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Wed Sep 19 23:15:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 45921180677 for ; Wed, 19 Sep 2018 23:15:04 +0200 (CEST) Received: (qmail 32268 invoked by uid 500); 19 Sep 2018 21:15:03 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 31438 invoked by uid 99); 19 Sep 2018 21:15:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Sep 2018 21:15:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 8A327C1A92 for ; Wed, 19 Sep 2018 21:15:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id OmcfFxUzZ4CL for ; Wed, 19 Sep 2018 21:15:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 741F25F590 for ; Wed, 19 Sep 2018 21:15:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 942E1E12FE for ; Wed, 19 Sep 2018 21:15:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 42B0623FA0 for ; Wed, 19 Sep 2018 21:15:00 +0000 (UTC) Date: Wed, 19 Sep 2018 21:15:00 +0000 (UTC) From: "Erik Krogen (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-15726) Create utility to limit frequency of log statements MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621221#comment-16621221 ] Erik Krogen commented on HADOOP-15726: -------------------------------------- Thanks [~vagarychen]! Can we get this backported to older branches as well? The current patch applies to {{branch-3.1}} directly, but not to {{branch-3.0}} because {{FSNamesystemLock}} uses log4j API instead of slf4j. I attached a {{branch-3.0}} patch which also works for {{branch-2}}. > Create utility to limit frequency of log statements > --------------------------------------------------- > > Key: HADOOP-15726 > URL: https://issues.apache.org/jira/browse/HADOOP-15726 > Project: Hadoop Common > Issue Type: Improvement > Components: common, util > Reporter: Erik Krogen > Assignee: Erik Krogen > Priority: Major > Fix For: 3.2.0 > > Attachments: HADOOP-15726.000.patch, HADOOP-15726.001.patch, HADOOP-15726.002.patch, HADOOP-15726.003.patch, HDFS-15726-branch-3.0.003.patch > > > There is a common pattern of logging a behavior that is normally extraneous. Under some circumstances, such a behavior becomes common, flooding the logs and making it difficult to see what else is going on in the system. Under such situations it is beneficial to limit how frequently the extraneous behavior is logged, while capturing some summary information about the suppressed log statements. > This is currently implemented in {{FSNamesystemLock}} (in HDFS-10713). We have additional use cases for this in HDFS-13791, so this is a good time to create a common utility for different sites to share this logic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org