hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10161) Add a method to change the default value of dmax in hadoop.properties
Date Tue, 17 Dec 2013 03:27:08 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13850059#comment-13850059
] 

Yang He commented on HADOOP-10161:
----------------------------------

Colin Patrick McCabe ,  it is really necessary, especially for hbase users. e.g. when a region
is splitted to two new ones, the old metrics of the region is invalid, but the metics are
still here ,they also store values to disk ,the invalid data will Increase disk pressure (io
speed,disk space and so on).
Just changing the default value could't meet our needs in some case:defferent types of  metric
may be configured different  collecting period,so they need diffirent dmax, we should  and
must easily configure them.
e.g:
Scenario 1: Config different default value for different metric types
#
datanode.sink.ganglia2.dmax_default=150
datanode.sink.ganglia2.context=dfs
datanode.sink.ganglia2.servers=10.28.171.212:8661

datanode.sink.ganglia3.dmax_default=130
datanode.sink.ganglia3.context=rpm
datanode.sink.ganglia3.servers=10.28.171.212:8661

Scenario 2:  Change the default value of dmax for all metrics
#
datanode.sink.ganglia.dmax_default=130
datanode.sink.ganglia.servers=10.28.171.212:8661

Scenario 3:  Keep the default value of dmax 
datanode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
datanode.sink.ganglia.period=10
datanode.sink.ganglia.servers=10.28.171.212:8661


> Add a method to change the default value of dmax in hadoop.properties
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-10161
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10161
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 2.2.0
>            Reporter: Yang He
>         Attachments: HADOOP-10161_0_20131211.patch, hadoop-metrics.properties, hadoop-metrics2.properties
>
>
> The property of dmax in ganglia is a configurable time to rotate metrics. Therefore,
no more value of the metric will be emit to the gmond, after 'dmax' seconds, then gmond will
destroy the metric in memory. In Hadoop metrics framework, the default value of 'dmax' is
0. It means the gmond will never destroy the metric although the metric is disappeared. The
gmetad daemon also does not delete the rrdtool file forever. 
> We need to add a method to configure the default value of dmax for all metrics in hadoop.properties.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message