hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7055) Add tracing to DFSInputStream
Date Mon, 22 Sep 2014 05:54:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142925#comment-14142925
] 

Colin Patrick McCabe commented on HDFS-7055:
--------------------------------------------

This patch adds tracing to DFSInputStream.  It's fairly granulary tracing, done at the level
of read operations.  I also added trace spans to the BlockReaders (both remote and local)
so that the amount of time they spend can be disentangled from other things.  I added tracing
to {{BlockSender}} so that the amount of time we spend sending the block can be seen.

By setting {{dfs.client.trace.sampler}}, users can make their HDFS client create new trace
spans on some requests.  The available options are {{NeverSampler}}, {{AlwaysSampler}}, and
{{ProbabilitySampler}}.

One thing to keep in mind here is that if you call {{Trace.startSpan}} with {{Sampler.NEVER}},
and there is an existing thread trace span, a subspan will always be created.  But if there
is no existing thread trace span, none will be created.  So there is no need for if statements
and null/not null logic here.  You can always get "pass through" behavior by using {{Sampler.NEVER}}.

> Add tracing to DFSInputStream
> -----------------------------
>
>                 Key: HDFS-7055
>                 URL: https://issues.apache.org/jira/browse/HDFS-7055
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>    Affects Versions: 2.6.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-7055.002.patch
>
>
> Add tracing to DFSInputStream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message