hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12332) Logging improvement suggestion for SampleStat function MinMax.add
Date Mon, 21 Aug 2017 15:32:00 GMT
Xu Zhao created HDFS-12332:
------------------------------

             Summary: Logging improvement suggestion for SampleStat function MinMax.add
                 Key: HDFS-12332
                 URL: https://issues.apache.org/jira/browse/HDFS-12332
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: metrics
    Affects Versions: 3.0.0-alpha4
            Reporter: Xu Zhao
            Priority: Minor


In order to debug performance anomlies, it is better to see when the max and min value get
updated in metrics.
For example, in our workload we would like to see the longest latency of a write block request
on DataNode.

The following metric updating code could possibly update the min/max value in SampleStat class:

{code:java}
    datanode.getMetrics().addWriteBlockOp(elapsed());
    datanode.getMetrics().incrWritesFromClient(peer.isLocal(), size);
{code}

Here I am attaching an attempt to patch it.
The patch just adds additional DEBUG level logging in the MinMax class and prints the new
value when it gets updated.

Please let me know what you think. For example, if this patch would introduce too much noises,
what is the best way to trace the read/writeBlock that has the minimum/maximum latency?
Any comments are appreciated!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message