hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yangping wu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2863) ResourceManager will shutdown when job's queue is empty
Date Fri, 14 Nov 2014 10:51:34 GMT

     [ https://issues.apache.org/jira/browse/YARN-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

yangping wu updated YARN-2863:
------------------------------
    Description: 
When I submit a job to hadoop cluster, but don't specified a queuename as follow
{code}
 $HADOOP_HOME"hadoop" jar statistics.jar com.wyp.Sts -Dmapreduce.job.queuename=   
 {code}
 and if *yarn.scheduler.fair.allow-undeclared-pools* is not overwrite by user(default is true),
then QueueManager will call createLeafQueue method to create the queue, but I didn't set the
mapreduce.job.queuename property. Then throw MetricsException
{code}
2014-11-14 16:07:57,358 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Error in handling event type APP_ADDED to the scheduler
org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root already exists!
        at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:126)
        at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:107)
        at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:217)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:94)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:57)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:57)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createLeafQueue(QueueManager.java:191)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:136)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:652)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:610)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1015)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
        at java.lang.Thread.run(Thread.java:744)
2014-11-14 16:07:57,359 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Exiting, bbye..
{code}

  was:
When I submit a job to hadoop cluster, but don't specified a queuename as follow
{code}
 $HADOOP_HOME"hadoop" jar statistics.jar com.wyp.Sts -Dmapreduce.job.queuename=   
 {code}
 and if {yarn.scheduler.fair.allow-undeclared-pools is not overwrite by user(default is true),
then QueueManager will call createLeafQueue method to create the queue, but I didn't set the
mapreduce.job.queuename property. Then throw MetricsException
{code}
2014-11-14 16:07:57,358 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Error in handling event type APP_ADDED to the scheduler
org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root already exists!
        at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:126)
        at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:107)
        at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:217)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:94)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:57)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:57)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createLeafQueue(QueueManager.java:191)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:136)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:652)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:610)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1015)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
        at java.lang.Thread.run(Thread.java:744)
2014-11-14 16:07:57,359 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Exiting, bbye..
{code}


> ResourceManager will shutdown when job's queue is empty
> -------------------------------------------------------
>
>                 Key: YARN-2863
>                 URL: https://issues.apache.org/jira/browse/YARN-2863
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.2.0
>            Reporter: yangping wu
>   Original Estimate: 8h
>  Remaining Estimate: 8h
>
> When I submit a job to hadoop cluster, but don't specified a queuename as follow
> {code}
>  $HADOOP_HOME"hadoop" jar statistics.jar com.wyp.Sts -Dmapreduce.job.queuename=   
>  {code}
>  and if *yarn.scheduler.fair.allow-undeclared-pools* is not overwrite by user(default
is true), then QueueManager will call createLeafQueue method to create the queue, but I didn't
set the mapreduce.job.queuename property. Then throw MetricsException
> {code}
> 2014-11-14 16:07:57,358 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Error in handling event type APP_ADDED to the scheduler
> org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root already
exists!
>         at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:126)
>         at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:107)
>         at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:217)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:94)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:57)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:57)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createLeafQueue(QueueManager.java:191)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:136)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:652)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:610)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1015)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
>         at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
>         at java.lang.Thread.run(Thread.java:744)
> 2014-11-14 16:07:57,359 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Exiting, bbye..
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message