Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1AAE910AA7 for ; Tue, 26 Nov 2013 21:49:11 +0000 (UTC) Received: (qmail 13251 invoked by uid 500); 26 Nov 2013 21:49:05 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 13184 invoked by uid 500); 26 Nov 2013 21:49:05 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 13176 invoked by uid 99); 26 Nov 2013 21:49:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Nov 2013 21:49:05 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=HTML_FONT_FACE_BAD,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of munnavarsk@gmail.com designates 209.85.212.182 as permitted sender) Received: from [209.85.212.182] (HELO mail-wi0-f182.google.com) (209.85.212.182) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Nov 2013 21:48:58 +0000 Received: by mail-wi0-f182.google.com with SMTP id en1so7783539wid.3 for ; Tue, 26 Nov 2013 13:48:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Ga3JzfO56qig9XEq5Aa2ri/K38fr/3SqeWYc86b9Rb0=; b=ptxMyxPto64my+cCxog7Dq1YhEubUKUgXkjbEDRszEs/7KcEmU9BL9novoSyXt4IGg PC56j8WLkuk4WGtCuuscJiHxULyvFAXbql2eoCcmjpMOUqhzkgqxYaRPCyCPbuBGGgyd 8IB1UaMydCuswvwHwTmcynlv7adVs7pWMiOBPlDCrOHRz+U7EhPk8xU6BIlSmgWYrKrt wx0zDOmkCt5QC4wb40FYx+gOw9Z4ZGyaprQWYoH3JPtJUXxmCWjCotTsYegM8u2JAow6 CSzgK1Uwtn+iq0/X6XIG/3Z/jHn/xGLYiTx1DuYy482CR0y5YYtCYbvmCUS2hXyKvSsT alcg== MIME-Version: 1.0 X-Received: by 10.180.149.175 with SMTP id ub15mr19970468wib.10.1385502518334; Tue, 26 Nov 2013 13:48:38 -0800 (PST) Received: by 10.194.250.2 with HTTP; Tue, 26 Nov 2013 13:48:38 -0800 (PST) In-Reply-To: References: Date: Wed, 27 Nov 2013 03:18:38 +0530 Message-ID: Subject: Re: Working with Capacity Scheduler From: Munna To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c3811870d95c04ec1b7117 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c3811870d95c04ec1b7117 Content-Type: text/plain; charset=ISO-8859-1 Hi, Can i able to get the solution on Capacity Scheduler ? On Wed, Nov 27, 2013 at 2:24 AM, Munna wrote: > yes... acl's are not enforced. > > > On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault > wrote: > >> Do you mean that acl are not being enforced? >> >> Olivier >> On 26 Nov 2013 20:18, "Munna" wrote: >> >>> Hi Olivier, >>> >>> have u hold any solution, why user are not identifying their queues? >>> >>> Thanks >>> Munna >>> >>> >>> On Tue, Nov 26, 2013 at 11:17 PM, Munna wrote: >>> >>>> *mapred queue -list* result: >>>> >>>> [user@host ~]$ mapred queue -list >>>> 13/11/26 09:37:48 INFO service.AbstractService: >>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. >>>> 13/11/26 09:37:48 INFO service.AbstractService: >>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. >>>> ====================== >>>> Queue Name : exploration >>>> Queue State : running >>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, >>>> CurrentCapacity: 0.0 >>>> ====================== >>>> Queue Name : a >>>> Queue State : running >>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, >>>> CurrentCapacity: 0.0 >>>> ====================== >>>> Queue Name : b >>>> Queue State : running >>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, >>>> CurrentCapacity: 0.0 >>>> ====================== >>>> Queue Name : c >>>> Queue State : running >>>> Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0, >>>> CurrentCapacity: 0.0 >>>> ====================== >>>> Queue Name : production >>>> Queue State : running >>>> Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0, >>>> CurrentCapacity: 0.0 >>>> >>>> and I saw in RM logs, mapred user try to find default queue, logs are >>>> below: >>>> >>>> 2013-11-26 05:22:59,804 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application >>>> with id 1 submitted by user mapred >>>> 2013-11-26 05:22:59,808 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred >>>> IP=10.10.10.1 OPERATION=Submit Application Request >>>> TARGET=ClientRMService RESULT=SUCCESS APPID=application_1385472086605_0001 >>>> 2013-11-26 05:22:59,828 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: >>>> application_1385472086605_0001 State change from NEW to SUBMITTED >>>> 2013-11-26 05:22:59,829 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: >>>> Registering appattempt_1385472086605_0001_000001 >>>> 2013-11-26 05:22:59,830 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: >>>> appattempt_1385472086605_0001_000001 State change from NEW to SUBMITTED >>>> 2013-11-26 05:22:59,832 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: >>>> appattempt_1385472086605_0001_000001 State change from SUBMITTED to FAILED >>>> 2013-11-26 05:22:59,836 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: >>>> application_1385472086605_0001 State change from SUBMITTED to FAILED >>>> 2013-11-26 05:22:59,837 WARN >>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred >>>> OPERATION=Application Finished - Failed TARGET=RMAppManager >>>> RESULT=FAILURE DESCRIPTION=App failed with state: FAILED >>>> PERMISSIONS=Application appattempt_1385472086605_0001_000001 submitted by >>>> user mapred to unknown queue: default APPID=application_1385472086605_0001 >>>> 2013-11-26 05:22:59,841 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: >>>> Removing info for app: application_1385472086605_0001 >>>> 2013-11-26 05:22:59,842 INFO >>>> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: >>>> appId=application_1385472086605_0001,name=select count(*) from >>>> crc_exchange_rate(Stage-1),user=mapred,queue=default,state=FAILED,trackingUrl=hostname:8088/proxy/application_1385472086605_0001/,appMasterHost=N/A,startTime=1385472179799,finishTime=1385472179836 >>>> >>>> >>>> >>>> On Tue, Nov 26, 2013 at 10:36 PM, Olivier Renault < >>>> orenault@hortonworks.com> wrote: >>>> >>>>> Could you confirm that the queue are running by: >>>>> mapred queue -list >>>>> >>>>> Otherwise what error do you get when submitting a job ? >>>>> >>>>> Thanks, >>>>> Olivier >>>>> >>>>> >>>>> On 26 November 2013 15:33, Munna wrote: >>>>> >>>>>> Below configured queue's are showing, then why i cannt able to run >>>>>> the job. >>>>>> >>>>>> [root@host ~]# hadoop queue -showacls >>>>>> DEPRECATED: Use of this script to execute mapred command is >>>>>> deprecated. >>>>>> Instead use the mapred command for it. >>>>>> >>>>>> 13/11/26 05:25:43 INFO service.AbstractService: >>>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. >>>>>> 13/11/26 05:25:44 INFO service.AbstractService: >>>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. >>>>>> Queue acls for user : root >>>>>> >>>>>> Queue Operations >>>>>> ===================== >>>>>> root ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>>> exploration ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>>> a ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>>> b ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>>> c ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>>> production ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Nov 26, 2013 at 8:05 PM, Olivier Renault < >>>>>> orenault@hortonworks.com> wrote: >>>>>> >>>>>>> Sorry my mistake, it's : >>>>>>> mapred queue -showacls >>>>>>> >>>>>>> Or you can also use hadoop queue -showacls as suggested by Jitendra >>>>>>> >>>>>>> Thanks, >>>>>>> Olviier >>>>>>> >>>>>>> >>>>>>> On 26 November 2013 14:17, Munna wrote: >>>>>>> >>>>>>>> Yes! I have executed same command: >>>>>>>> >>>>>>>> [user@host ~]$* yarn queue -showacls* >>>>>>>> >>>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: queue >>>>>>>> Caused by: java.lang.ClassNotFoundException: queue >>>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>>>>>> at java.security.AccessController.doPrivileged(Native >>>>>>>> Method) >>>>>>>> at >>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>>>>> at >>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>>>>> Could not find the main class: queue. Program will exit. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Nov 26, 2013 at 7:25 PM, Olivier Renault < >>>>>>>> orenault@hortonworks.com> wrote: >>>>>>>> >>>>>>>>> it should be : >>>>>>>>> yarn queue -showacls >>>>>>>>> >>>>>>>>> Olivier >>>>>>>>> >>>>>>>>> >>>>>>>>> On 26 November 2013 13:28, Munna wrote: >>>>>>>>> >>>>>>>>>> Hi Olivier, >>>>>>>>>> >>>>>>>>>> Thank you for your reply. >>>>>>>>>> >>>>>>>>>> As you said, i ran those commands and i am getting following >>>>>>>>>> error message on both the commands. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *[root@host ~]# sudo -u yarn yarn queue -showacls Exception in >>>>>>>>>> thread "main" java.lang.NoClassDefFoundError: queueCaused by: >>>>>>>>>> java.lang.ClassNotFoundException: queue at >>>>>>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:202) at >>>>>>>>>> java.security.AccessController.doPrivileged(Native Method) at >>>>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) at >>>>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:306) at >>>>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at >>>>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the >>>>>>>>>> main class: queue. Program will exit.* >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Munna >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Nov 26, 2013 at 4:10 PM, Olivier Renault < >>>>>>>>>> orenault@hortonworks.com> wrote: >>>>>>>>>> >>>>>>>>>>> Could you maybe send us the output of : >>>>>>>>>>> - yarn queue -showacls >>>>>>>>>>> - yarn queue -list >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Olivier >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 26 November 2013 05:53, Munna wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I working with Capacity Scheduler on YARN and I have configured >>>>>>>>>>>> different queues. I can able to see all the queues on RM ui. But, when i >>>>>>>>>>>> start to run MR jobs with configured user names(yarn,mapred), i am unable >>>>>>>>>>>> to run the Jobs and job are suspended. Again i set default as FIFO working >>>>>>>>>>>> fine. >>>>>>>>>>>> >>>>>>>>>>>> Can you please help me out sort this issue and configured >>>>>>>>>>>> Configurations are given below. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> * >>>>>>>>>>>> yarn.scheduler.capacity.root.queues >>>>>>>>>>>> production,exploration >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.queues >>>>>>>>>>>> a,b,c >>>>>>>>>>>> yarn.scheduler.capacity.root.capacity >>>>>>>>>>>> 100 >>>>>>>>>>>> yarn.scheduler.capacity.root.production.capacity >>>>>>>>>>>> 70 >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.capacity >>>>>>>>>>>> 30 >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.a.capacity >>>>>>>>>>>> 30 >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.b.capacity >>>>>>>>>>>> 30 >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.c.capacity >>>>>>>>>>>> 40 yarn.scheduler.capacity.root. >>>>>>>>>>>> production.acl_submit_applications >>>>>>>>>>>> yarn,mapred userb >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.a.acl_submit_applications >>>>>>>>>>>> >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.b.acl_submit_applications >>>>>>>>>>>> userb >>>>>>>>>>>> yarn.scheduler.capacity.root.exploration.c.acl_submit_applications >>>>>>>>>>>> userc* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Regards* >>>>>>>>>>>> >>>>>>>>>>>> *Munna* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>> CONFIDENTIALITY NOTICE >>>>> NOTICE: This message is intended for the use of the individual or >>>>> entity to which it is addressed and may contain information that is >>>>> confidential, privileged and exempt from disclosure under applicable law. >>>>> If the reader of this message is not the intended recipient, you are hereby >>>>> notified that any printing, copying, dissemination, distribution, >>>>> disclosure or forwarding of this communication is strictly prohibited. If >>>>> you have received this communication in error, please contact the sender >>>>> immediately and delete it from your system. Thank You. >>>>> >>>> >>>> >>>> >>>> -- >>>> *Regards* >>>> >>>> *Munna* >>>> >>> >>> >>> >>> -- >>> *Regards* >>> >>> *Munna* >>> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> > > > > -- > *Regards* > > *Munna* > -- *Regards* *Munna* --001a11c3811870d95c04ec1b7117 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,

Can i able to get the solution on Capacity= Scheduler ?


On Wed, Nov 27, 2013 at 2:24 AM, Munna &= lt;munnavarsk@gma= il.com> wrote:
yes... acl's are not en= forced.


On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault <orenault@hortonwo= rks.com> wrote:

Do you mean that acl are not = being enforced?

Olivier

On 26 Nov 2013 20:18, "Munna" <munnavarsk@gmail.com<= /a>> wrote:
Hi Olivier,

have u hold any solutio= n, why user are not identifying their queues?

Thanks
<= /div>Munna


On Tue, Nov 26, 2013 at 11:17 PM, Munna <munnavarsk@gmail.com> wrote:
mapred queue -list result:

[user@host ~= ]$ mapred queue -list
13/11/26 09:37:48 INFO service.AbstractService: Se= rvice:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/11/26 0= 9:37:48 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client= .YarnClientImpl is started.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Queue= Name : exploration
Queue State : running
Scheduling Info : Capacity:= 30.000002, MaximumCapacity: 1.0, CurrentCapacity: 0.0
=A0=A0=A0 =3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=A0=A0=A0 Q= ueue Name : a
=A0=A0=A0 Queue State : running
=A0=A0=A0 Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, Curr= entCapacity: 0.0
=A0=A0=A0 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D
=A0=A0=A0 Queue Name : b
=A0=A0=A0 Queue State = : running
=A0=A0=A0 Scheduling Info : Capacity: 30.000002, MaximumCapaci= ty: 1.0, CurrentCapacity: 0.0
=A0=A0=A0 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D
=A0=A0=A0 Queue Name : c
=A0=A0=A0 Queue State : running
=A0= =A0=A0 Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0, CurrentCapac= ity: 0.0
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D
Queue Name : production
Queue State : running
Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0, CurrentCapacity: 0.= 0

and I saw in RM logs, mapred user try to find default queue,= logs are below:

2013-11-26 05:22:59,804 INFO org.apache.hadoop.yarn= .server.resourcemanager.ClientRMService: Application with id 1 submitted by= user mapred
2013-11-26 05:22:59,808 INFO org.apache.hadoop.yarn.server.resourcemanager.= RMAuditLogger: USER=3Dmapred=A0=A0 IP=3D10.10.10.1=A0 OPERATION=3DSubmit Ap= plication Request=A0=A0=A0 TARGET=3DClientRMService=A0 RESULT=3DSUCCESS=A0 = APPID=3Dapplication_1385472086605_0001
2013-11-26 05:22:59,828 INFO org.apache.hadoop.yarn.server.resourcemanager.= rmapp.RMAppImpl: application_1385472086605_0001 State change from NEW to SU= BMITTED
2013-11-26 05:22:59,829 INFO org.apache.hadoop.yarn.server.resou= rcemanager.ApplicationMasterService: Registering appattempt_1385472086605_0= 001_000001
2013-11-26 05:22:59,830 INFO org.apache.hadoop.yarn.server.resourcemanager.= rmapp.attempt.RMAppAttemptImpl: appattempt_1385472086605_0001_000001 State = change from NEW to SUBMITTED
2013-11-26 05:22:59,832 INFO org.apache.had= oop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_= 1385472086605_0001_000001 State change from SUBMITTED to FAILED
2013-11-26 05:22:59,836 INFO org.apache.hadoop.yarn.server.resourcemanager.= rmapp.RMAppImpl: application_1385472086605_0001 State change from SUBMITTED= to FAILED
2013-11-26 05:22:59,837 WARN org.apache.hadoop.yarn.server.re= sourcemanager.RMAuditLogger: USER=3Dmapred=A0=A0 OPERATION=3DApplication Fi= nished - Failed TARGET=3DRMAppManager=A0=A0=A0=A0 RESULT=3DFAILURE=A0 DESCR= IPTION=3DApp failed with state: FAILED=A0=A0=A0=A0=A0=A0 PERMISSIONS=3DAppl= ication appattempt_1385472086605_0001_000001 submitted by user mapred to un= known queue: default APPID=3Dapplication_1385472086605_0001
2013-11-26 05:22:59,841 INFO org.apache.hadoop.yarn.server.resourcemanager.= recovery.RMStateStore: Removing info for app: application_1385472086605_000= 1
2013-11-26 05:22:59,842 INFO org.apache.hadoop.yarn.server.resourceman= ager.RMAppManager$ApplicationSummary: appId=3Dapplication_1385472086605_000= 1,name=3Dselect count(*) from crc_exchange_rate(Stage-1),user=3Dmapred,queu= e=3Ddefault,state=3DFAILED,trackingUrl=3Dhostname:8088/proxy/application_13= 85472086605_0001/,appMasterHost=3DN/A,startTime=3D1385472179799,finishTime= =3D1385472179836



On Tue, Nov 26, 2013 at 10:36 PM, Olivier Renault = <orenault@= hortonworks.com> wrote:
Could you confirm that the = queue are running by:
mapred queue -list

Other= wise what error do you get when submitting a job ?

Thanks,=A0
Oliv= ier


On 26 Novembe= r 2013 15:33, Munna <munnavarsk@gmail.com> wrote:
Below configured queue's are showing, then why i cannt= able to run the job.

[root@host ~]# hadoop queue -showacls
DEPRE= CATED: Use of this script to execute mapred command is deprecated.
Inste= ad use the mapred command for it.

13/11/26 05:25:43 INFO service.AbstractService: Service:org.apache.hado= op.yarn.client.YarnClientImpl is inited.
13/11/26 05:25:44 INFO service.= AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is st= arted.
Queue acls for user :=A0 root

Queue=A0 Operations
=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
root=A0 ADMINISTER_QUEU= E,SUBMIT_APPLICATIONS
exploration=A0 ADMINISTER_QUEUE,SUBMIT_APPLICATION= S
a=A0 ADMINISTER_QUEUE,SUBMIT_APPLICATIONS
b=A0 ADMINISTER_QUEUE,SUBMIT_APPLICATIONS
c=A0 ADMINISTER_QUEUE,SUBMIT_A= PPLICATIONS
production=A0 ADMINISTER_QUEUE,SUBMIT_APPLICATIONS



On Tue, Nov 26, 2013 at 8:05 PM, Olivier Renault <orenault@hortonwo= rks.com> wrote:
Sorry my mistake, it's = :=A0
mapred queue -showacls

Or you can also us= e hadoop queue -showacls as suggested by=A0Jitendra

Thanks,=A0<= /div>
Olviier


On 26 November 2013 14:17, Munna <munnavarsk@gmail.com> wrote:
Yes! I have executed same command:
[user@host ~]$ yarn queue -showacls

Exception in thread "main" java.lang.NoClassDefFoundError: qu= eue
Caused by: java.lang.ClassNotFoundException: queue
=A0=A0=A0=A0=A0=A0=A0 at java.net.URLClassLoader$1.run(URLClassLoader.java:= 202)
=A0=A0=A0=A0=A0=A0=A0 at java.security.AccessController.doPrivilege= d(Native Method)
=A0=A0=A0=A0=A0=A0=A0 at java.net.URLClassLoader.findCl= ass(URLClassLoader.java:190)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.ClassLoa= der.loadClass(ClassLoader.java:306)
=A0=A0=A0=A0=A0=A0=A0 at sun.misc.Launcher$AppClassLoader.loadClass(Launche= r.java:301)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.ClassLoader.loadClass(Cla= ssLoader.java:247)
Could not find the main class: queue.=A0 Program will= exit.



On Tue, Nov 26, 2013 at 7:25 PM, Olivier= Renault <orenault@hortonworks.com> wrote:
it should be :
yarn queue -showacls

Olivier
=


On 26 November 2013 13:28, Munna <munn= avarsk@gmail.com> wrote:
Hi Olivier,

Thank you for your reply.

As you said, i ran those commands an= d i am getting following error message on both the commands.

[root@host ~]# sudo -u yarn yarn queue -showacls
Exception in thread "main" java.lang.NoClassDefFoundError: queue<= br>Caused by: java.lang.ClassNotFoundException: queue
=A0=A0=A0=A0=A0=A0= =A0 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
=A0=A0=A0= =A0=A0=A0=A0 at java.security.AccessController.doPrivileged(Native Method)<= br> =A0=A0=A0=A0=A0=A0=A0 at java.net.URLClassLoader.findClass(URLClassLoader.j= ava:190)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.ClassLoader.loadClass(ClassL= oader.java:306)
=A0=A0=A0=A0=A0=A0=A0 at sun.misc.Launcher$AppClassLoade= r.loadClass(Launcher.java:301)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.ClassL= oader.loadClass(ClassLoader.java:247)
Could not find the main class: queue.=A0 Program will exit.

Regards,
Munna



On Tue, Nov 26, 2013 at 4:10 PM, Olivier Renault <orenault@hortonwo= rks.com> wrote:
Could you maybe send us the o= utput of :
=A0- yarn queue -showacls
=A0- yarn queue -list

Thanks,=A0
Olivier


On 26 November 2013 05:53, Munna <munnavarsk@gmail.com> w= rote:
Hi,

I working with Capacity Schedul= er on YARN and I have configured different queues. I can able to see all th= e queues on RM ui. But, when i start to run MR jobs with configured user na= mes(yarn,mapred), i am unable to run the Jobs and job are suspended. Again = i set default as FIFO working fine.

Can you please help me out sort this issue and configured Configu= rations are given below.

<property>
=A0=A0=A0 <name&g= t;yarn.scheduler.capacity.root.queues</name>
=A0=A0=A0 <value&g= t;production,exploration</value>
</property>
<property>
=A0=A0=A0 <name>yarn.schedul= er.capacity.root.exploration.queues</name>
=A0=A0=A0 <value>= a,b,c</value>
</property>

<property>
=A0=A0= =A0 <name>yarn.scheduler.capacity.root.capacity</name>
=A0=A0=A0 <value>100</value>
</property>

<pr= operty>
=A0=A0=A0 <name>yarn.scheduler.capacity.root.production= .capacity</name>
=A0=A0=A0 <value>70</value>
</p= roperty>
<property>
=A0=A0=A0 <name>yarn.scheduler.capacity.root.expl= oration.capacity</name>
=A0=A0=A0 <value>30</value></property>

<property>
=A0=A0=A0 <name>yarn.sc= heduler.capacity.root.exploration.a.capacity</name>
=A0=A0=A0 <value>30</value>
</property>

<pro= perty>
=A0=A0=A0 <name>yarn.scheduler.capacity.root.exploration= .b.capacity</name>
=A0=A0=A0 <value>30</value>
<= /property>

<property>
=A0=A0=A0 <name>yarn.scheduler.capacity.root.= exploration.c.capacity</name>
=A0=A0=A0 <value>40</value&= gt;
</property>


<property>
=A0 <name>yar= n.scheduler.capacity.root. production.acl_submit_applications</name><= br> =A0 <value>yarn,mapred</value>
</property>

<= property>
=A0=A0=A0 <value>userb</value>
=A0=A0=A0 <= ;name>yarn.scheduler.capacity.root.exploration.a.acl_submit_applications= </name>
</property>

<property>
=A0=A0=A0 <name>yarn.sch= eduler.capacity.root.exploration.b.acl_submit_applications</name>
= =A0=A0=A0 <value>userb</value>
</property>

<= property>
=A0=A0=A0 <name>yarn.scheduler.capacity.root.exploration.c.acl_submit= _applications</name>
=A0=A0=A0 <value>userc</value></property>

</configuration>


--
Regards

Munna



<= /blockquote>
<= /div>

CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.



--
Regards

Munna<= /span>



--
Regards

Munna<= /span>

CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.


--
Regards

Munna<= span>


--
Regards

Munna=
--001a11c3811870d95c04ec1b7117--