hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4486) Add log category for long-running DFSClient notices
Date Fri, 15 Aug 2014 19:09:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098950#comment-14098950
] 

Zhe Zhang commented on HDFS-4486:
---------------------------------

My current plan is to follow this page to generate an extended logger class: http://logging.apache.org/log4j/2.x/manual/customloglevels.html.
I haven't seen these kind of extended logger classes in the Hadoop project though. So please
let me know if you think there's a better approach.

> Add log category for long-running DFSClient notices
> ---------------------------------------------------
>
>                 Key: HDFS-4486
>                 URL: https://issues.apache.org/jira/browse/HDFS-4486
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Todd Lipcon
>            Assignee: Zhe Zhang
>            Priority: Minor
>
> There are a number of features in the DFS client which are transparent but can make a
fairly big difference for performance -- two in particular are short circuit reads and native
checksumming. Because we don't want log spew for clients like "hadoop fs -cat" we currently
log only at DEBUG level when these features are disabled. This makes it difficult to troubleshoot/verify
for long-running perf-sensitive clients like HBase.
> One simple solution is to add a new log category - eg o.a.h.h.DFSClient.PerformanceAdvisory
- which long-running clients could enable at DEBUG level without getting the full debug spew.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message