ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-5607) Yarn Nodemanager Metrics only update every few minutes
Date Tue, 06 May 2014 18:33:16 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matt Foley updated AMBARI-5607:
-------------------------------

    Description: 
Yarn Nodemanager Metrics take far too long between updates.

To demonstrate:

Run Terasort or anything that runs mapreduce:
{code}
hdfs dfs -mkdir -p benchmarks/terasort
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapred.map.tasks=72
-Dmapred.reduce.tasks=36  1000000 benchmarks/terasort/input
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort -Dmapred.map.tasks=72
-Dmapred.reduce.tasks=36 benchmarks/terasort/input benchmarks/terasort/output
hdfs dfs -rm -R -skipTrash benchmarks/terasort
{code}
Then repeatedly probe the API at:
{noformat}http://<server>:8081/api/v1/clusters/c1/services/YARN/components/NODEMANAGER?fields=host_components/metrics/yarn{noformat}
It usually takes 2-3 minutes to see the metrics update, very repeatable.


  was:
Yarn Nodemanager Metrics take far too long between updates.

To demonstrate:

Run Terasort or anything that runs mapreduce:
{code}
hdfs dfs -mkdir -p benchmarks/terasort
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapred.map.tasks=72
-Dmapred.reduce.tasks=36  1000000 benchmarks/terasort/input
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort -Dmapred.map.tasks=72
-Dmapred.reduce.tasks=36 benchmarks/terasort/input benchmarks/terasort/output
hdfs dfs -rm -R -skipTrash benchmarks/terasort
{code}
Then repeatedly probe 
{noformat}http://<server>:8081/api/v1/clusters/c1/services/YARN/components/NODEMANAGER?fields=host_components/metrics/yarn{noformat}
It usually takes 2-3 minutes to see the metrics update, very repeatable.



> Yarn Nodemanager Metrics only update every few minutes
> ------------------------------------------------------
>
>                 Key: AMBARI-5607
>                 URL: https://issues.apache.org/jira/browse/AMBARI-5607
>             Project: Ambari
>          Issue Type: Bug
>    Affects Versions: 1.5.1
>         Environment: sles11sp1
>            Reporter: Michael Harp
>
> Yarn Nodemanager Metrics take far too long between updates.
> To demonstrate:
> Run Terasort or anything that runs mapreduce:
> {code}
> hdfs dfs -mkdir -p benchmarks/terasort
> hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapred.map.tasks=72
-Dmapred.reduce.tasks=36  1000000 benchmarks/terasort/input
> hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort -Dmapred.map.tasks=72
-Dmapred.reduce.tasks=36 benchmarks/terasort/input benchmarks/terasort/output
> hdfs dfs -rm -R -skipTrash benchmarks/terasort
> {code}
> Then repeatedly probe the API at:
> {noformat}http://<server>:8081/api/v1/clusters/c1/services/YARN/components/NODEMANAGER?fields=host_components/metrics/yarn{noformat}
> It usually takes 2-3 minutes to see the metrics update, very repeatable.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message