hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Zhuge (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10822) Log DataNodes in the write pipeline
Date Thu, 01 Sep 2016 19:10:21 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

John Zhuge updated HDFS-10822:
------------------------------
    Attachment: HDFS-10822.001.patch

Patch 001:
* {{initDataStreaming}} is a better place to add DEBUG msg because the log msg will have {{file}},
{{block}}, and {{pipeline}} all together.
* No unit test because the patch adds DEBUG msg only.

{{TestDataStream}} log that shows the new DEBUG msg:
{noformat}
2016-09-01 11:54:18,470 [DataStreamer for file /file1 block BP-1484500550-172.16.1.255-1472756056162:blk_1073741825_1001]
DEBUG hdfs.DataStreamer (DataStreamer.java:initDataStreaming(512)) - nodes [DatanodeInfoWithStorage[127.0.0.1:61765,DS-124a1018-6033-4f81-bc43-cba740bd9538,DISK]]
storageTypes [DISK] storageIDs [DS-124a1018-6033-4f81-bc43-cba740bd9538]
{noformat}

> Log DataNodes in the write pipeline
> -----------------------------------
>
>                 Key: HDFS-10822
>                 URL: https://issues.apache.org/jira/browse/HDFS-10822
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 2.6.0
>            Reporter: John Zhuge
>            Assignee: John Zhuge
>            Priority: Trivial
>              Labels: supportability
>         Attachments: HDFS-10822.001.patch
>
>
> Trying to diagnose a slow HDFS flush, taking longer than 10 seconds, but did not know
which DNs were involved in the write pipeline. Of course, I could search NN log for the list
of DNs, but it might not be possible sometimes or convenient.
> Propose to add a DEBUG trace to DataStreamer#setPipeline to print the list of DNs in
the pipeline.
> BTW, we do print the list of DNs during pipeline recovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message