hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonger (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file
Date Wed, 15 Nov 2017 01:32:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16252804#comment-16252804
] 

Yonger edited comment on HADOOP-14475 at 11/15/17 1:31 AM:
-----------------------------------------------------------

For the # 2 point of my update
----------------------------------
without this update,the metrics be flush into file like this(each record with same URI mark
the same "context+registry name" as begin):

{code:java}
1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
{code}


with this update, each record will be easy to distinguish:

{code:java}
1510631110244 s3afs.S3aFileSystemMetrics: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3aFileSystemMetrics2: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3aFileSystemMetrics3: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
{code}
 


was (Author: iyonger):
For the # 2 point of my update
----------------------------------
without this update,the metrics be flush into file like this(each record mark the same "context+registry
name" as begin):

{code:java}
1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
{code}


with this update, each record will be easy to distinguish:

{code:java}
1510631110244 s3afs.S3aFileSystemMetrics: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3aFileSystemMetrics2: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
1510631110244 s3afs.S3aFileSystemMetrics3: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs,
fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx
{code}
 

> Metrics of S3A don't print out  when enable it in Hadoop metrics property file
> ------------------------------------------------------------------------------
>
>                 Key: HADOOP-14475
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14475
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 2.8.0
>         Environment: uname -a
> Linux client01 4.4.0-74-generic #95-Ubuntu SMP Wed Apr 12 09:50:34 UTC 2017 x86_64 x86_64
x86_64 GNU/Linux
>  cat /etc/issue
> Ubuntu 16.04.2 LTS \n \l
>            Reporter: Yonger
>            Assignee: Yonger
>         Attachments: HADOOP-14475-003.patch, HADOOP-14475.002.patch, HADOOP-14475.005.patch,
HADOOP-14475.006.patch, HADOOP-14475.008.patch, HADOOP-14775.007.patch, failsafe-report-s3a-it.html,
failsafe-report-s3a-scale.html, failsafe-report-scale.html, failsafe-report-scale.zip, s3a-metrics.patch1,
stdout.zip
>
>
> *.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink
> #*.sink.file.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink
> #*.sink.influxdb.url=http:/xxxxxxxxxx
> #*.sink.influxdb.influxdb_port=8086
> #*.sink.influxdb.database=hadoop
> #*.sink.influxdb.influxdb_username=hadoop
> #*.sink.influxdb.influxdb_password=hadoop
> #*.sink.ingluxdb.cluster=c1
> *.period=10
> #namenode.sink.influxdb.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink
> #S3AFileSystem.sink.influxdb.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink
> S3AFileSystem.sink.file.filename=s3afilesystem-metrics.out
> I can't find the out put file even i run a MR job which should be used s3.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message