hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4486) Add log category for long-running DFSClient notices
Date Thu, 14 Aug 2014 18:10:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097308#comment-14097308
] 

Zhe Zhang commented on HDFS-4486:
---------------------------------

[~tlipcon] I wonder if you can share a little more details on the short circuit read and native
checksumming use cases. What particular metrics are of interest (e.g. IOPS)? Depending on
that, it might make sense to make it a JMX metric instead of free text logging.

> Add log category for long-running DFSClient notices
> ---------------------------------------------------
>
>                 Key: HDFS-4486
>                 URL: https://issues.apache.org/jira/browse/HDFS-4486
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Todd Lipcon
>            Assignee: Zhe Zhang
>            Priority: Minor
>
> There are a number of features in the DFS client which are transparent but can make a
fairly big difference for performance -- two in particular are short circuit reads and native
checksumming. Because we don't want log spew for clients like "hadoop fs -cat" we currently
log only at DEBUG level when these features are disabled. This makes it difficult to troubleshoot/verify
for long-running perf-sensitive clients like HBase.
> One simple solution is to add a new log category - eg o.a.h.h.DFSClient.PerformanceAdvisory
- which long-running clients could enable at DEBUG level without getting the full debug spew.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message