hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Surendra Singh Lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10303) DataStreamer#ResponseProcessor calculate packet acknowledge duration wrongly.
Date Mon, 18 Apr 2016 14:11:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245720#comment-15245720
] 

Surendra Singh Lilhore commented on HDFS-10303:
-----------------------------------------------

I am getting the "Slow ReadProcessor read" log in my cluster when I increase the socket timeout
for client.

{noformat}
16/04/14 17:57:59 WARN DataStreamer: Slow ReadProcessor read fields for block BP-873267638-192.168.100.12-1460002479721:blk_1073752739_11917
took 47858ms (threshold=30000ms); ack: seqno: 3 reply: SUCCESS reply: SUCCESS reply: SUCCESS
downstreamAckTimeNanos: 803180 flag: 0 flag: 0 flag: 0, targets: [DatanodeInfoWithStorage[192.168.100.9:25009,DS-d552bfd7-1c38-430d-8703-c3b539caf351,DISK],
DatanodeInfoWithStorage[192.168.100.11:25009,DS-02897c9b-bceb-4790-b08a-f711d8e3fd81,DISK],
DatanodeInfoWithStorage[192.168.100.10:25009,DS-fae7b497-a269-4614-afe5-7006660eafcf,DISK]]
{noformat}

But when I checked the packet send time, it is same as packet acknowledge time

{noformat}
16/04/14 17:57:59 DEBUG DataStreamer: DataStreamer block BP-873267638-192.168.100.12-1460002479721:blk_1073752739_11917
sending packet packet seqno: 3 offsetInBlock: 8704 lastPacketInBlock: false lastByteOffsetInBlock:
12316
{noformat}




This is coming because {{ResponseProcessor}} set the current time as begin time and wait for
the packet ack, after getting the ack it will calculate the duration and compare with the
{{dfs.client.slow.io.warning.threshold.ms}}.

{code}
          // read an ack from the pipeline
          long begin = Time.monotonicNow();
          ack.readFields(blockReplyStream);
          long duration = Time.monotonicNow() - begin;
{code}

Suppose client sent two packets and now he doesn't have data to write, after some time he
got more data and sent third packet.

Client waited for some time after sending second packet. Time between second packet and third
packet should not be considered by {{ResponseProcessor}} in packet acknowledge duration.


> DataStreamer#ResponseProcessor calculate packet acknowledge duration wrongly.
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-10303
>                 URL: https://issues.apache.org/jira/browse/HDFS-10303
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.7.2
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>
> Packets acknowledge duration should be calculated based on the packet send time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message