hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8088) Reduce the number of HTrace spans generated by HDFS reads
Date Thu, 09 Apr 2015 01:42:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486519#comment-14486519

Colin Patrick McCabe commented on HDFS-8088:

* Shorten the method names we're tracing.  {{ClientProtocol#create}} instead of {{org.apache.hadoop.hdfs.protocol.ClientProtocol.create}},

* Don't create trace spans for {{DFSInputStream#read(final byte buf[], int off, int len)}}
and {{int read(final ByteBuffer buf)}}.  Note that we still create trace spans inside the
block readers, when refilling the block reader buffers.

* RemoteBlockReader2.java: include the block ID as a key/value annotation, not in the name
of the trace span itself.

* TestTracing.java: explain which trace spans we couldn't find, if we can't find some trace
spans we are looking for.

> Reduce the number of HTrace spans generated by HDFS reads
> ---------------------------------------------------------
>                 Key: HDFS-8088
>                 URL: https://issues.apache.org/jira/browse/HDFS-8088
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-8088.001.patch
> HDFS generates too many trace spans on read right now.  Every call to read() we make
generates its own span, which is not very practical for things like HBase or Accumulo that
do many such reads as part of a single operation.  Instead of tracing every call to read(),
we should only trace the cases where we refill the buffer inside a BlockReader.

This message was sent by Atlassian JIRA

View raw message