hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From #TANG SHANJIANG# <STA...@e.ntu.edu.sg>
Subject MapReduce Job running Problems with queue designation in fair scheduler for Yarn-2.2.0.
Date Fri, 06 Dec 2013 03:05:46 GMT
Hi,
   I encounter a problem with Yarn's fair scheduler. The thing is that,  I first set a queue
by configuring fair-scheduler.xml below. Next I try to submit a job to that queue by designating
queue name via "mapreduce.job.queuename= amelie".
fair-scheduler.xml:
<allocations>
  <queue name="amelie">
    <minResources>10000 mb,1vcores</minResources>
    <maxResources>90000 mb,1vcores</maxResources>
    <maxRunningApps>50</maxRunningApps>
    <weight>2.0</weight>
    <schedulingPolicy>fair</schedulingPolicy>
  </queue>
</allocations>
However, the job is stuck in that queue without running, and the yarn-amelie-resourcemanager-pdcbl3-log
file shows  errors that,
2013-12-06 10:16:20,551 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:amelie (auth:SIMPLE) cause:org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException:
Application with id 'application_1386295642784_0001' doesn't exist in RM.
2013-12-06 10:16:20,552 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8032, call
org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 155.69.148.21:60571
Call#1416 Retry#0: error: org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException:
Application with id 'application_1386295642784_0001' doesn't exist in RM.
org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: Application with id 'application_1386295642784_0001'
doesn't exist in RM.
        at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:247)
        at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:120)
        at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:241)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)

Moreover, the strange point is that, if I don't set the queue 'amelie' in fair-scheduler.xml
and run my job directly. The problem can run successfully by creating a queue called 'amelie'
automatically. It seems that Yarn's fair scheduler do not support the job running on user's
queue configuration in fair-scheduler.xml


Mime
View raw message