hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Williams <...@joetify.com>
Subject Re: Hadoop and Ganglia Meterics
Date Thu, 24 Jul 2008 20:15:38 GMT
Ah, yeah, I found that one. :) Patching 
'java/org/apache/hadoop/mapred/JobInProgress.java' on 0.17.1.

-joe


Jason Venner wrote:
> I have only applied this patch as far forward as 0.16.0
>
> Joe Williams wrote:
>> Sweet, thanks.
>>
>>
>> Jason Venner wrote:
>>> Once the patch is applied you should start seeing the ganglia metrics
>>>
>>> We do.
>>>
>>>
>>> Joe Williams wrote:
>>>> Once I have the patch applied and have it running should I see the 
>>>> metrics? Or do I need to additional work?
>>>>
>>>> Thanks.
>>>> -Joe
>>>>
>>>>
>>>> Jason Venner wrote:
>>>>> I applied the patch in the jira to my distro
>>>>>
>>>>> Joe Williams wrote:
>>>>>> Thanks Jason, until this is implemented are how are you pulling 
>>>>>> stats from Hadoop?
>>>>>>
>>>>>> -joe
>>>>>>
>>>>>>
>>>>>> Jason Venner wrote:
>>>>>>> Check out
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/HADOOP-3422
>>>>>>>
>>>>>>>
>>>>>>> Joe Williams wrote:
>>>>>>>> I have been attempting to get Hadoop metrics in Ganliga and

>>>>>>>> have been unsuccessful thus far. I have see this thread 
>>>>>>>> (http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200712.mbox/raw/%3C475E4FFA.7050007@yahoo-inc.com%3E/)

>>>>>>>> but it didn't help much.
>>>>>>>>
>>>>>>>> I have setup my properties file like so:
>>>>>>>>
>>>>>>>>> [root@hadoop-head-node01 current]# cat 
>>>>>>>>> conf/hadoop-metrics.properties
>>>>>>>>> dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>>>>>>>>> dfs.period=10
>>>>>>>>> dfs.servers=127.0.0.1:8649
>>>>>>>>>
>>>>>>>>> mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>>>>>>>>> mapred.period=10
>>>>>>>>> mapred.servers=127.0.0.1:8649
>>>>>>>>
>>>>>>>> And if I 'telnet 127.0.0.1  8649' I receive the Ganglia XML

>>>>>>>> metrics output without any hadoop specific metrics:
>>>>>>>>
>>>>>>>>> [root@hadoop-head-node01 current]# telnet 127.0.0.1 
8649
>>>>>>>>> Trying 127.0.0.1...
>>>>>>>>> Connected to localhost (127.0.0.1).
>>>>>>>>> Escape character is '^]'.
>>>>>>>>> ?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>
>>>>>>>>> !DOCTYPE GANGLIA_XML [
>>>>>>>>>   !ELEMENT GANGLIA_XML (GRID|CLUSTER|HOST)*>
>>>>>>>>>       !ATTLIST GANGLIA_XML VERSION CDATA #REQUIRED>
>>>>>>>>>       !ATTLIST GANGLIA_XML SOURCE CDATA #REQUIRED>
>>>>>>>>> --SNIP--
>>>>>>>>
>>>>>>>> Is there more I need to do to get the metrics to show up
in 
>>>>>>>> this output, am I doing something incorrectly? Do I need
to 
>>>>>>>> have a gmetric script run in a cron to update the stats?
If so, 
>>>>>>>> does anyone have a hadoop specific example of this?
>>>>>>>>
>>>>>>>> Any info would be helpful.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>> -Joe
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>

-- 
Name: Joseph A. Williams
Email: joe@joetify.com


Mime
View raw message