hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benyi Wang <bewang.t...@gmail.com>
Subject Re: Questions about hadoop-metrics2.properties
Date Fri, 25 Oct 2013 17:56:24 GMT
Thanks, Luke. Your answer is really helpful.


On Wed, Oct 23, 2013 at 11:51 PM, Luke Lu <llu@apache.org> wrote:

> 1. File and Ganglia are the only bundled sinks, though there are
> socket/json (for chukwa) and graphite sinks patches in the works.
> 2. Hadoop metrics (and metrics2) is mostly designed for system/process
> metrics, which means you'll need to attach jconsole to your map/reduce task
> processes to see your task metrics instrumented via metrics. What you
> actually want is probably custom job counters.
> 3. You don't need any configuration to use JMX to access metrics2, as JMX
> is currently on by default. The configuration in hadoop-metrics2.properties
> is mostly for optional sink configuration and metrics filtering.
>
> __Luke
>
>
>
> On Wed, Oct 23, 2013 at 4:21 PM, Benyi Wang <bewang.tech@gmail.com> wrote:
>
>> 1. Does hadoop metrics2 only support File and Ganglia sink?
>> 2. Can I expose metrics as JMX, especially for customized metrics? I
>> created some  metrics in my mapreduce job and could successfully output
>> them using a FileSink. But if I use jconsole to access YARN nodemanager, I
>> can only see hadoop metrics e.g Hadoop/NodeManager/NodeManagerMetrices
>> etc.,  not mine with prefix maptask. How to setup to see maptask/reducetask
>> prefix metrics?
>> 3. Is there an example using jmx? I could not find
>>
>> The configuration syntax is:
>>
>>   [prefix].[source|sink|jmx|].[instance].[option]
>>
>>
>> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html
>>
>
>

Mime
View raw message