hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8088) Reduce the number of HTrace spans generated by HDFS reads
Date Tue, 14 Apr 2015 20:25:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494799#comment-14494799

Billie Rinaldi commented on HDFS-8088:

I would also lean towards goal #1, figuring out what parts of an operation took the longest.
 I have it on my to-do list to analyze the spans Accumulo is creating and cut down on the
ones that don't seem to be adding much information.  I think targeting 2.7.1 for changes in
HDFS tracing will be fine for Accumulo.  I've made our sampling configurable and made our
span receiver able to configurably drop 0ms spans to handle any excess of spans from HDFS
tracing.  (I know dropping some of the spans in a trace is not a recommended procedure, but
it ends up being okay since nearly all 0ms spans have no children.)

> Reduce the number of HTrace spans generated by HDFS reads
> ---------------------------------------------------------
>                 Key: HDFS-8088
>                 URL: https://issues.apache.org/jira/browse/HDFS-8088
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-8088.001.patch
> HDFS generates too many trace spans on read right now.  Every call to read() we make
generates its own span, which is not very practical for things like HBase or Accumulo that
do many such reads as part of a single operation.  Instead of tracing every call to read(),
we should only trace the cases where we refill the buffer inside a BlockReader.

This message was sent by Atlassian JIRA

View raw message