hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15348) S3A Input Stream bytes read counter isn't getting through to StorageStatistics/insturmentation properly
Date Tue, 27 Mar 2018 19:29:00 GMT
Steve Loughran created HADOOP-15348:
---------------------------------------

             Summary: S3A Input Stream bytes read counter isn't getting through to StorageStatistics/insturmentation
properly
                 Key: HADOOP-15348
                 URL: https://issues.apache.org/jira/browse/HADOOP-15348
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.0.0, 3.1.0
            Reporter: Steve Loughran


TL;DR: we should have common storage statistics for bytes read and bytes written, and S3A
should use them in its instrumentation and have enum names to match.

# in the S3AInputStream we call {{S3AInstrumentation.StreamStatistics.bytesRead(long)}}, which
adds the amount to {{bytesRead}}, in a read(), readFully, or forward seek() reading in data
# and in {{S3AInstrumentation.mergeInputStreamStatistics}}, that is pulled into streamBytesRead.
# which has a Statistics name of ""stream_bytes_read"
# but that is served up in the Storage statistics as "STREAM_SEEK_BYTES_READ", which is the
wrong name.
# and there isn't a common name for the counter across other filesystems.

For now: people can use the wrong name in the enum; we may want to think about retaining it
when adding the correct name. And maybe add a @Evolving/@LimitedPrivate scope pair to the
enum



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message