hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-14973) Log StorageStatistics
Date Mon, 23 Oct 2017 16:20:00 GMT
Sean Mackrory created HADOOP-14973:

             Summary: Log StorageStatistics
                 Key: HADOOP-14973
                 URL: https://issues.apache.org/jira/browse/HADOOP-14973
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/s3
            Reporter: Sean Mackrory
            Assignee: Sean Mackrory

S3A is currently storing much more detailed metrics via StorageStatistics than are logged
in a MapReduce job. Eventually, it would be nice to get Spark, MapReduce and other workloads
to retrieve and store these metrics, but it may be some time before they all do that. I'd
like to consider having S3A publish the metrics itself in some form. This is tricky, as S3A
has no daemon but lives inside various other processes.

Perhaps writing to a log file at some configurable interval and on close() would be the best
we could do. Other ideas would be welcome.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message