hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongwook Kwon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3843) Fair Scheduler should not accept apps with space keys as queue name
Date Mon, 22 Jun 2015 23:28:00 GMT
Dongwook Kwon created YARN-3843:
-----------------------------------

             Summary: Fair Scheduler should not accept apps with space keys as queue name
                 Key: YARN-3843
                 URL: https://issues.apache.org/jira/browse/YARN-3843
             Project: Hadoop YARN
          Issue Type: Bug
          Components: fairscheduler
    Affects Versions: 2.5.0, 2.4.0
            Reporter: Dongwook Kwon
            Priority: Minor


As YARN-461, since empty string queue name is not valid, queue name with space keys such as
" " ,"   " should not be accepted either, also not as prefix nor postfix. 
e.g) "root.test.queuename  ", or "root.test. queuename"

I have 2 specific cases kill RM with these space keys as part of queue name.
1) Without placement policy (hadoop 2.4.0 and above), 
When a job is submitted with " "(space key) as queue name
e.g) mapreduce.job.queuename=" "

2) With placement policy (hadoop 2.5.0 and above)
 Once a job is submitted without space key as queue name, and submit another job with space
key.
e.g) 1st time: mapreduce.job.queuename="root.test.user1" 
2nd time: mapreduce.job.queuename="root.test.user1 "

{code}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.974 sec <<< FAILURE!
- in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
testQueueNameWithSpace(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler)
 Time elapsed: 0.724 sec  <<< ERROR!
org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root,q1=adhoc,q2=birvine
already exists!
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:135)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:112)
	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:218)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:96)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:56)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:66)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createQueue(QueueManager.java:169)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getQueue(QueueManager.java:120)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:88)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:660)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:569)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1127)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueNameWithSpace(TestFairScheduler.java:627)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message