hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Yan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2863) ResourceManager will shutdown when job's queuename is empty
Date Mon, 17 Nov 2014 18:48:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214973#comment-14214973
] 

Wei Yan commented on YARN-2863:
-------------------------------

[~397090770], in the hadoop trunk, this problem looks already solved.
If:
{code}
bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi -Dmapreduce.job.queuename=
20 10
{code}

The job will be rejected:
{code}
java.io.IOException: Failed to run job : Reject application application_1416249736343_0002
submitted by user root with an empty queue name.
	at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:445)
{code}

> ResourceManager will shutdown when job's queuename is empty
> -----------------------------------------------------------
>
>                 Key: YARN-2863
>                 URL: https://issues.apache.org/jira/browse/YARN-2863
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.2.0
>            Reporter: yangping wu
>   Original Estimate: 8h
>  Remaining Estimate: 8h
>
> When I submit a job to hadoop cluster, but don't specified a queuename as follow
> {code}
>  $HADOOP_HOME/bin/hadoop jar statistics.jar com.iteblog.Sts -Dmapreduce.job.queuename=
  
>  {code}
>  and if *yarn.scheduler.fair.allow-undeclared-pools* is not overwrite by user(default
is true), then QueueManager will call createLeafQueue method to create the queue, because
mapreduce.job.queuename is empty and cann't find it in QueueManager  .But  this  will throw
MetricsException
> {code}
> 2014-11-14 16:07:57,358 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Error in handling event type APP_ADDED to the scheduler
> org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root already
exists!
>         at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:126)
>         at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:107)
>         at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:217)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:94)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:57)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:57)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createLeafQueue(QueueManager.java:191)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:136)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:652)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:610)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1015)
>         at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
>         at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
>         at java.lang.Thread.run(Thread.java:744)
> 2014-11-14 16:07:57,359 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager:
Exiting, bbye..
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message