hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Tianyi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9081) False-positive of ACK slow log in DFSClient
Date Tue, 15 Sep 2015 10:23:45 GMT
He Tianyi created HDFS-9081:

             Summary: False-positive of ACK slow log in DFSClient
                 Key: HDFS-9081
                 URL: https://issues.apache.org/jira/browse/HDFS-9081
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.6.0
            Reporter: He Tianyi
            Assignee: He Tianyi
            Priority: Minor

This issue is related with code below:
if (duration > dfsclientSlowLogThresholdMs
    && ack.getSeqno() != Packet.HEART_BEAT_SEQNO) {
      .warn("Slow ReadProcessor read fields took " + duration
          + "ms (threshold=" + dfsclientSlowLogThresholdMs + "ms); ack: "
          + ack + ", targets: " + Arrays.asList(targets));
} else if (DFSClient.LOG.isDebugEnabled()) {
  DFSClient.LOG.debug("DFSClient " + ack);

DFSClient prints slow log when awaited after unexpected amount of time (usually 30000 ms).
This is a good indicator for network or I/O performance issue.

However, there is scenario that this slow log is false-positive, i.e. a reducer, (StageA)
iterates over records with identical key, this takes arbitrary amount of time, but generates
no output. (StageB) Then, it output arbitrary number of records when meet a different key.

If one StageA lasts more than 30000 ms (as the example above), there will be one or more slow
log generated, which is not related to any HDFS performance issue. 
In general cases, user should not expect this, as they could be misguided.

This message was sent by Atlassian JIRA

View raw message