hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13845) s3a to instrument duration of HTTP calls
Date Fri, 21 Jul 2017 03:06:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095706#comment-16095706

Yonger commented on HADOOP-13845:

Another usage is push http duration into database(such as influxdb), and get a realtime show
via grafana. that you can know the duration of one kind(path) of request when is longest when
suffering some bottleneck, and then you can do trouble shooting work in the range of higher
duration time. which can not be done if you just only output the final result in FS stats.

And what's going on for HADOOP-13844? i expect to put http duration into s3a metrics. 

> s3a to instrument duration of HTTP calls
> ----------------------------------------
>                 Key: HADOOP-13845
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13845
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Priority: Minor
> HADOOP-13844 proposes pulling out the swift duration classes for reuse; this patch proposes
instrumenting s3a with it.
> One interesting question: what to do with the values. For now, they could just be printed,
but it might be interesting to include in FS stats collected at the end of a run. However,
those are all assumed to be simple counters where merging is a matter of addition. These are
more metrics

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message