Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 29 Oct 2012 21:58:12 +0000 (UTC)
From: "Suresh Srinivas (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <1435240131.41156.1351547892545.JavaMail.jiratomcat@arcas>
In-Reply-To: <1289290901.67246.1350610563933.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (HDFS-4080) Add an option to disable block-level
 state change logging
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486408#comment-13486408 ] 

Suresh Srinivas commented on HDFS-4080:
---------------------------------------

Kihwal, quick comment, having come out of my own log cleanup work :-). It is probably a good idea to retain "Block *" and not change to "Dir *". BTW I have removed unnecessary prefixes such "NameNode." from the logs - see HDFS-4122. Sorry you will have to rebase this patch.

                
> Add an option to disable block-level state change logging
> ---------------------------------------------------------
>
>                 Key: HDFS-4080
>                 URL: https://issues.apache.org/jira/browse/HDFS-4080
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 3.0.0, 2.0.3-alpha
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>         Attachments: hdfs-4080.1.patch, hdfs-4080.patch
>
>
> Although the block-level logging in namenode is useful for debugging, it can add a significant overhead to busy hdfs clusters since they are done while the namespace write lock is held. One example is shown in HDFS-4075. In this example, the write lock was held for 5 minutes while logging 11 million log messages for 5.5 million block invalidation events. 
> It will be useful if we have an option to disable these block-level log messages and keep other state change messages going.  If others feel that they can turned into DEBUG (with addition of isDebugEnabled() checks), that may also work too, but there might be people depending on the messages.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira