hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9576) HTrace: collect position/length information on read operations
Date Wed, 13 Jan 2016 02:29:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095489#comment-15095489

Colin Patrick McCabe commented on HDFS-9576:

Cool.  This sounds useful for profiling applications.

    scope.addKVAnnotation("position", Long.toString(pos));
Can we use "pos" instead?  It would help keep span sizes down.

Hmm.  I'm curious whether we need to separate request length and returned length, or if we
can just have one length, which is the length actually read.  It seems like if you see a lot
of really short lengths, you can do some deeper digging to figure out why... but I'm not sure
if it's worth carrying this info on every read trace span.

> HTrace: collect position/length information on read operations
> --------------------------------------------------------------
>                 Key: HDFS-9576
>                 URL: https://issues.apache.org/jira/browse/HDFS-9576
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client, tracing
>    Affects Versions: 2.7.1
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-9576.00.patch, HDFS-9576.01.patch, HDFS-9576.02.patch, HDFS-9576.03.patch,
HDFS-9576.04.patch, HDFS-9576.05.patch
> HTrace currently collects the path of each read operation (both stateful and position
reads). To better understand applications' I/O behavior, it is also useful to track the position
and length of read operations.

This message was sent by Atlassian JIRA

View raw message