Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 86E02EB91 for ; Wed, 30 Jan 2013 21:07:39 +0000 (UTC) Received: (qmail 66928 invoked by uid 500); 30 Jan 2013 21:07:33 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 66821 invoked by uid 500); 30 Jan 2013 21:07:33 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 66807 invoked by uid 99); 30 Jan 2013 21:07:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jan 2013 21:07:33 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,NO_RDNS_DOTCOM_HELO,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: 98.139.253.105 is neither permitted nor denied by domain of daryn@yahoo-inc.com) Received: from [98.139.253.105] (HELO mrout2-b.corp.bf1.yahoo.com) (98.139.253.105) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jan 2013 21:07:27 +0000 Received: from sp1-ex07cas01.ds.corp.yahoo.com (sp1-ex07cas01.ds.corp.yahoo.com [216.252.116.137]) by mrout2-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id r0UL6gpx085899 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL); Wed, 30 Jan 2013 13:06:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1359580004; bh=uvfpM6EE2vF7LMgkGEHcJfdjIMvqJ+pgPmwEo3i9TYA=; h=From:To:CC:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=Am5dRwBahEp9oopDhzWVz8iMJLvlx4QqXMwHG8JM4LwYvO5xbazAR70VDp1lzoqII Zkj6pfhmI1z3ME4drg+AH6xqoS5NrOHz3z39Uj8da1Hd20uuOfb7l56gdx2/0upyo7 fuxfQT8QzOAm3dxdc4jSZKMx2AJ7xylTZ3H5bgE8= Received: from SP1-EX07VS02.ds.corp.yahoo.com ([216.252.116.135]) by sp1-ex07cas01.ds.corp.yahoo.com ([216.252.116.137]) with mapi; Wed, 30 Jan 2013 13:06:42 -0800 From: Daryn Sharp To: "user@hadoop.apache.org" CC: "user@oozie.apache.org" Date: Wed, 30 Jan 2013 13:06:39 -0800 Subject: Re: Oozie workflow error - renewing token issue Thread-Topic: Oozie workflow error - renewing token issue Thread-Index: Ac3/LbOw3tVQWowvQl6IAsRp3yifxw== Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_B54B22AC36764A70B4D539D805538251yahooinccom_" MIME-Version: 1.0 X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 580003002 X-Virus-Checked: Checked by ClamAV on apache.org --_000_B54B22AC36764A70B4D539D805538251yahooinccom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable The token renewer needs to be the job tracker principal. I think oozie had= "mr token" hardcoded at one point, but later changed it to use a conf sett= ing. The rest of the log looks very odd - ie. it looks like security is off, but= it can't be. It's trying to renew hdfs tokens issued for the "hdfs" super= user. Are you running your job as hdfs? With security enabled, the contai= ner executor won't run jobs for hdfs=85 It's also trying to renew as "mapr= ed (auth:SIMPLE)". SIMPLE means that security is disabled, which doesn't m= ake sense since tokens require kerberos. The cannot find renewer class for MAPREDUCE_DELEGATION_TOKEN is also odd be= cause it did subsequently try to renew the token. Daryn On Jan 30, 2013, at 2:14 PM, Alejandro Abdelnur wrote: Cobert, [Moving thread to user@oozie.a.o, BCCing common-user@hadoop.a.o] * What version of Oozie are you using? * Is the cluster a secure setup (Kerberos enabled)? * Would you mind posting the complete launcher logs? Thx On Wed, Jan 30, 2013 at 6:14 AM, Corbett Martin > wrote: Thanks for the tip. The sqoop command listed in the stdout log file is: sqoop import --driver org.apache.derby.jdbc.ClientDriver --connect jdbc:derby://test-server:1527/mondb --username monuser --password xxxxxxxxx --table MONITOR --split-by request_id --target-dir /mon/import --append --incremental append --check-column request_id --last-value 2000000 The following information is from the sterr and stdout log files. /var/log/hadoop-0.20-mapreduce/userlogs/job_201301231648_0029/attempt_20130= 1231648_0029_m_000000_0 # more stderr No such sqoop tool: sqoop. See 'sqoop help'. Intercepting System.exit(1) Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMai= n], exit code [1] /var/log/hadoop-0.20-mapreduce/userlogs/job_201301231648_0029/attempt_20130= 1231648_0029_m_000000_0 # more stdout ... ... ... >>> Invoking Sqoop command line now >>> 1598 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has no= t been set in the environment. Cannot check for additional configuration. Intercepting System.exit(1) <<< Invocation of Main class completed <<< Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMai= n], exit code [1] Oozie Launcher failed, finishing Hadoop job gracefully Oozie Launcher ends ~Corbett Martin Software Architect AbsoluteAR Accounts Receivable Services - An NHIN Solution -----Original Message----- From: Harsh J [mailto:harsh@cloudera.com] Sent: Tuesday, January 29, 2013 10:11 PM To: > Subject: Re: Oozie workflow error - renewing token issue The job that Oozie launches for your action, which you are observing is fai= ling, does its own logs (task logs) show any errors? On Wed, Jan 30, 2013 at 4:59 AM, Corbett Martin > wrote: > Oozie question > > > > I'm trying to run an Oozie workflow (sqoop action) from the Hue > console and it fails every time. No exception in the oozie log but I > see this in the Job Tracker log file. > > > > Two primary issues seem to be > > 1. Client mapred tries to renew a token with renewer specified as mr to= ken > > > > And > > > > 2. Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN > > > > Any ideas how to get past this? > > > > Full Stacktrace: > > > > 2013-01-29 17:11:28,860 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecret= Manager: > Creating password for identifier: owner=3Dhdfs, renewer=3Dmr token, > realUser=3Doozie, issueDate=3D1359501088860, maxDate=3D1360105888860, > sequenceNumber=3D75, masterKeyId=3D8 > > 2013-01-29 17:11:28,871 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecret= Manager: > Creating password for identifier: owner=3Dhdfs, renewer=3Dmr token, > realUser=3Doozie, issueDate=3D1359501088871, maxDate=3D1360105888871, > sequenceNumber=3D76, masterKeyId=3D8 > > 2013-01-29 17:11:29,202 INFO > org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: > registering token for renewal for service =3D10.204.12.62:8021 and jobID > =3D > job_201301231648_0029 > > 2013-01-29 17:11:29,211 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecret= Manager: > Token renewal requested for identifier: owner=3Dhdfs, renewer=3Dmr token, > realUser=3Doozie, issueDate=3D1359501088871, maxDate=3D1360105888871, > sequenceNumber=3D76, masterKeyId=3D8 > > 2013-01-29 17:11:29,211 ERROR > org.apache.hadoop.security.UserGroupInformation: > PriviledgedActionException as:mapred (auth:SIMPLE) > cause:org.apache.hadoop.security.AccessControlException: Client mapred > tries to renew a token with renewer specified as mr token > > 2013-01-29 17:11:29,211 WARN org.apache.hadoop.security.token.Token: > Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN > > 2013-01-29 17:11:29,211 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 9 on 8021, call renewDelegationToken(Kind: > MAPREDUCE_DELEGATION_TOKEN, Service: 10.204.12.62:8021, Ident: 00 04 > 68 64 > 66 73 08 6d 72 20 74 6f 6b 65 6e 05 6f 6f 7a 69 65 8a 01 3c 88 94 58 > 67 8a > 01 3c ac a0 dc 67 4c 08), rpc version=3D2, client version=3D28, > methodsFingerPrint=3D1830206421 from 10.204.12.62:9706: error: > org.apache.hadoop.security.AccessControlException: Client mapred tries > to renew a token with renewer specified as mr token > > org.apache.hadoop.security.AccessControlException: Client mapred tries > to renew a token with renewer specified as mr token > > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec > retManager.renewToken(AbstractDelegationTokenSecretManager.java:274) > > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.ja > va:3738) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > ava:39) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > orImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call > (WritableRpcEngine.java:474) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898) > > at > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693) > > at > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689) > > at java.security.AccessController.doPrivileged(Native > Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat > ion.java:1332) > > at > org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687) > > 2013-01-29 17:11:29,212 ERROR > org.apache.hadoop.security.UserGroupInformation: > PriviledgedActionException as:mapred (auth:SIMPLE) > cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.Ac= cessControlException): > Client mapred tries to renew a token with renewer specified as mr > token > > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec > retManager.renewToken(AbstractDelegationTokenSecretManager.java:274) > > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.ja > va:3738) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > ava:39) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > orImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call > (WritableRpcEngine.java:474) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898) > > at > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693) > > at > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689) > > at java.security.AccessController.doPrivileged(Native > Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat > ion.java:1332) > > at > org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687) > > > > 2013-01-29 17:11:29,212 WARN org.apache.hadoop.security.token.Token: > Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN > > 2013-01-29 17:11:29,212 ERROR > org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: > Exception renewing tokenKind: MAPREDUCE_DELEGATION_TOKEN, Service: > 10.204.12.62:8021, > Ident: 00 04 68 64 66 73 08 6d 72 20 74 6f 6b 65 6e 05 6f 6f 7a 69 65 > 8a 01 3c 88 94 58 67 8a 01 3c ac a0 dc 67 4c 08. Not rescheduled > > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessCo= ntrolException): > Client mapred tries to renew a token with renewer specified as mr > token > > at > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec > retManager.renewToken(AbstractDelegationTokenSecretManager.java:274) > > at > org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.ja > va:3738) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > ava:39) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > orImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call > (WritableRpcEngine.java:474) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898) > > at > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693) > > at > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689) > > at java.security.AccessController.doPrivileged(Native > Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat > ion.java:1332) > > at > org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687) > > > > at org.apache.hadoop.ipc.Client.call(Client.java:1160) > > at > org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngi > ne.java:225) > > at > org.apache.hadoop.mapred.$Proxy12.renewDelegationToken(Unknown > Source) > > at > org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:541) > > at > org.apache.hadoop.security.token.Token.renew(Token.java:372) > > at > org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$Rene > walTimerTask$1.run(DelegationTokenRenewal.java:218) > > at > org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$Rene > walTimerTask$1.run(DelegationTokenRenewal.java:214) > > at java.security.AccessController.doPrivileged(Native > Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat > ion.java:1332) > > at > org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$Rene > walTimerTask.run(DelegationTokenRenewal.java:213) > > at java.util.TimerThread.mainLoop(Timer.java:512) > > at java.util.TimerThread.run(Timer.java:462) > > > > ~Corbett Martin > > Software Architect > > AbsoluteAR Accounts Receivable Services - An NHIN Solution > > > > > ________________________________ > This message and its contents (to include attachments) are the > property of National Health Systems, Inc. and may contain confidential > and proprietary information. This email and any files transmitted with > it are intended solely for the use of the individual or entity to whom th= ey are addressed. > You are hereby notified that any unauthorized disclosure, copying, or > distribution of this message, or the taking of any unauthorized action > based on information contained herein is strictly prohibited. > Unauthorized use of information contained herein may subject you to > civil and criminal prosecution and penalties. If you are not the > intended recipient, you should delete this message immediately and > notify the sender immediately by telephone or by replying to this transmi= ssion. -- Harsh J This message and its contents (to include attachments) are the property of = National Health Systems, Inc. and may contain confidential and proprietary = information. This email and any files transmitted with it are intended sole= ly for the use of the individual or entity to whom they are addressed. You = are hereby notified that any unauthorized disclosure, copying, or distribut= ion of this message, or the taking of any unauthorized action based on info= rmation contained herein is strictly prohibited. Unauthorized use of inform= ation contained herein may subject you to civil and criminal prosecution an= d penalties. If you are not the intended recipient, you should delete this = message immediately and notify the sender immediately by telephone or by re= plying to this transmission. -- Alejandro --_000_B54B22AC36764A70B4D539D805538251yahooinccom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable The token renewer needs to= be the job tracker principal.  I think oozie had "mr token" hardcoded= at one point, but later changed it to use a conf setting.

The rest of the log looks very odd - ie. it looks like security is = off, but it can't be.  It's trying to renew hdfs tokens issued for the= "hdfs" superuser.  Are you running your job as hdfs?  With secur= ity enabled, the container executor won't run jobs for hdfs=85  It's a= lso trying to renew as "mapred (auth:SIMPLE)".  SIMPLE means that secu= rity is disabled, which doesn't make sense since tokens require kerberos.

The cannot find renewer class for MAPREDUCE_DE= LEGATION_TOKEN is also odd because it did subsequently try to renew the tok= en.

Daryn



On Jan 30, 2013, at 2:14 PM, Alejandro Abdelnur wrote:

Cobert,

[Moving thread to use= r@oozie.a.o, BCCing common-user@hadoop.a.o]

=
* What version of Oozie are you using?
* Is the cluster a secure setup (Kerberos enabled)?
* Would you mind posting the complete launcher logs?
<= div style=3D"">
Thx



On Wed= , Jan 30, 2013 at 6:14 AM, Corbett Martin <comartin@nhin.com> wrote:
Thanks for the tip.

The sqoop command listed in the stdout log file is:
                    sqoop=
                    impor= t
                    --dri= ver
                    org.a= pache.derby.jdbc.ClientDriver
                    --con= nect
                    jdbc:= derby://test-server:1527/mondb
                    --use= rname
                    monus= er
                    --pas= sword
                    xxxxx= xxxx
                    --tab= le
                    MONIT= OR
                    --spl= it-by
                    reque= st_id
                    --tar= get-dir
                    /mon/= import
                    --app= end
                    --inc= remental
                    appen= d
                    --che= ck-column
                    reque= st_id
                    --las= t-value
                    20000= 00

The following information is from the sterr and stdout log files.

/var/log/hadoop-0.20-mapreduce/userlogs/job_201301231648_0029/attempt_20130= 1231648_0029_m_000000_0 # more stderr
No such sqoop tool: sqoop. See 'sqoop help'.
Intercepting System.exit(1)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMai= n], exit code [1]


/var/log/hadoop-0.20-mapreduce/userlogs/job_201301231648_0029/attempt_20130= 1231648_0029_m_000000_0 # more stdout
...
...
...
>>> Invoking Sqoop command line now >>>

1598 [main] WARN  org.apache.sqoop.tool.SqoopTool  - $SQOOP_CONF_= DIR has not been set in the environment. Cannot check for additional config= uration.
Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMai= n], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher ends


~Corbett Martin
Software Architect
AbsoluteAR Accounts Receivable Services - An NHIN Solution

-----Original Message----- From: Harsh J [mailto:harsh@cloudera.= com]
Sent: Tuesday, January 29, 2013 10:11 PM
To: <user@hadoop.apache.org>
Subject: Re: Oozie workflow error - renewing token issue

The job that Oozie launches for your action, which you are observing is fai= ling, does its own logs (task logs) show any errors?

On Wed, Jan 30, 2013 at 4:59 AM, Corbett Martin <comartin@nhin.com> wrote:
> Oozie question
>
>
>
> I'm trying to run an Oozie workflow (sqoop action) from the Hue
> console and it fails every time.  No exception in the oozie log b= ut I
> see this in the Job Tracker log file.
>
>
>
> Two primary issues seem to be
>
> 1.   Client mapred tries to renew a token with renewer specified = as mr token
>
>
>
> And
>
>
>
> 2.   Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN<= br> >
>
>
> Any ideas how to get past this?
>
>
>
> Full Stacktrace:
>
>
>
> 2013-01-29 17:11:28,860 INFO
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec= retManager:
> Creating password for identifier: owner=3Dhdfs, renewer=3Dmr token, > realUser=3Doozie, issueDate=3D1359501088860, maxDate=3D1360105888860,<= br> > sequenceNumber=3D75, masterKeyId=3D8
>
> 2013-01-29 17:11:28,871 INFO
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec= retManager:
> Creating password for identifier: owner=3Dhdfs, renewer=3Dmr token, > realUser=3Doozie, issueDate=3D1359501088871, maxDate=3D1360105888871,<= br> > sequenceNumber=3D76, masterKeyId=3D8
>
> 2013-01-29 17:11:29,202 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal:
> registering token for renewal for service =3D10.204.12.62:8021 and jobID
> =3D
> job_201301231648_0029
>
> 2013-01-29 17:11:29,211 INFO
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec= retManager:
> Token renewal requested for identifier: owner=3Dhdfs, renewer=3Dmr tok= en,
> realUser=3Doozie, issueDate=3D1359501088871, maxDate=3D1360105888871,<= br> > sequenceNumber=3D76, masterKeyId=3D8
>
> 2013-01-29 17:11:29,211 ERROR
> org.apache.hadoop.security.UserGroupInformation:
> PriviledgedActionException as:mapred (auth:SIMPLE)
> cause:org.apache.hadoop.security.AccessControlException: Client mapred=
> tries to renew a token with renewer specified as mr token
>
> 2013-01-29 17:11:29,211 WARN org.apache.hadoop.security.token.Token: > Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN
>
> 2013-01-29 17:11:29,211 INFO org.apache.hadoop.ipc.Server: IPC Server<= br> > handler 9 on 8021, call renewDelegationToken(Kind:
> MAPREDUCE_DELEGATION_TOKEN, Service: 10.204.12.62:8021, Ident: 00 04
> 68 64
> 66 73 08 6d 72 20 74 6f 6b 65 6e 05 6f 6f 7a 69 65 8a 01 3c 88 94 58 > 67 8a
> 01 3c ac a0 dc 67 4c 08), rpc version=3D2, client version=3D28,
> methodsFingerPrint=3D1830206421 from 10.204.12.62:9706: error:
> org.apache.hadoop.security.AccessControlException: Client mapred tries=
> to renew a token with renewer specified as mr token
>
> org.apache.hadoop.security.AccessControlException: Client mapred tries=
> to renew a token with renewer specified as mr token
>
>           at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec=
> retManager.renewToken(AbstractDelegationTokenSecretManager.java:274) >
>           at
> org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.ja=
> va:3738)
>
>           at sun.reflect.NativeMethodAccessor= Impl.invoke0(Native
> Method)
>
>           at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j=
> ava:39)
>
>           at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess=
> orImpl.java:25)
>
>           at java.lang.reflect.Method.invoke(= Method.java:597)
>
>           at
> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call=
> (WritableRpcEngine.java:474)
>
>           at org.apache.hadoop.ipc.RPC$Server= .call(RPC.java:898)
>
>           at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
>
>           at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
>
>           at java.security.AccessController.d= oPrivileged(Native
> Method)
>
>           at javax.security.auth.Subject.doAs= (Subject.java:396)
>
>           at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat=
> ion.java:1332)
>
>           at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
>
> 2013-01-29 17:11:29,212 ERROR
> org.apache.hadoop.security.UserGroupInformation:
> PriviledgedActionException as:mapred (auth:SIMPLE)
> cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security= .AccessControlException):
> Client mapred tries to renew a token with renewer specified as mr
> token
>
>           at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec=
> retManager.renewToken(AbstractDelegationTokenSecretManager.java:274) >
>           at
> org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.ja=
> va:3738)
>
>           at sun.reflect.NativeMethodAccessor= Impl.invoke0(Native
> Method)
>
>           at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j=
> ava:39)
>
>           at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess=
> orImpl.java:25)
>
>           at java.lang.reflect.Method.invoke(= Method.java:597)
>
>           at
> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call=
> (WritableRpcEngine.java:474)
>
>           at org.apache.hadoop.ipc.RPC$Server= .call(RPC.java:898)
>
>           at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
>
>           at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
>
>           at java.security.AccessController.d= oPrivileged(Native
> Method)
>
>           at javax.security.auth.Subject.doAs= (Subject.java:396)
>
>           at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat=
> ion.java:1332)
>
>           at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
>
>
>
> 2013-01-29 17:11:29,212 WARN org.apache.hadoop.security.token.Token: > Cannot find class for token kind MAPREDUCE_DELEGATION_TOKEN
>
> 2013-01-29 17:11:29,212 ERROR
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal:
> Exception renewing tokenKind: MAPREDUCE_DELEGATION_TOKEN, Service:
> 10.204.12.62:8= 021,
> Ident: 00 04 68 64 66 73 08 6d 72 20 74 6f 6b 65 6e 05 6f 6f 7a 69 65<= br> > 8a 01 3c 88 94 58 67 8a 01 3c ac a0 dc 67 4c 08. Not rescheduled
>
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.Acces= sControlException):
> Client mapred tries to renew a token with renewer specified as mr
> token
>
>           at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSec=
> retManager.renewToken(AbstractDelegationTokenSecretManager.java:274) >
>           at
> org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.ja=
> va:3738)
>
>           at sun.reflect.NativeMethodAccessor= Impl.invoke0(Native
> Method)
>
>           at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j=
> ava:39)
>
>           at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess=
> orImpl.java:25)
>
>           at java.lang.reflect.Method.invoke(= Method.java:597)
>
>           at
> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call=
> (WritableRpcEngine.java:474)
>
>           at org.apache.hadoop.ipc.RPC$Server= .call(RPC.java:898)
>
>           at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
>
>           at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
>
>           at java.security.AccessController.d= oPrivileged(Native
> Method)
>
>           at javax.security.auth.Subject.doAs= (Subject.java:396)
>
>           at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat=
> ion.java:1332)
>
>           at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
>
>
>
>           at org.apache.hadoop.ipc.Client.cal= l(Client.java:1160)
>
>           at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngi=
> ne.java:225)
>
>           at
> org.apache.hadoop.mapred.$Proxy12.renewDelegationToken(Unknown
> Source)
>
>           at
> org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:541) >
>           at
> org.apache.hadoop.security.token.Token.renew(Token.java:372)
>
>           at
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$Rene=
> walTimerTask$1.run(DelegationTokenRenewal.java:218)
>
>           at
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$Rene=
> walTimerTask$1.run(DelegationTokenRenewal.java:214)
>
>           at java.security.AccessController.d= oPrivileged(Native
> Method)
>
>           at javax.security.auth.Subject.doAs= (Subject.java:396)
>
>           at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat=
> ion.java:1332)
>
>           at
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$Rene=
> walTimerTask.run(DelegationTokenRenewal.java:213)
>
>           at java.util.TimerThread.mainLoop(T= imer.java:512)
>
>           at java.util.TimerThread.run(Timer.= java:462)
>
>
>
> ~Corbett Martin
>
> Software Architect
>
> AbsoluteAR Accounts Receivable Services - An NHIN Solution
>
>
>
>
> ________________________________
> This message and its contents (to include attachments) are the
> property of National Health Systems, Inc. and may contain confidential=
> and proprietary information. This email and any files transmitted with=
> it are intended solely for the use of the individual or entity to whom= they are addressed.
> You are hereby notified that any unauthorized disclosure, copying, or<= br> > distribution of this message, or the taking of any unauthorized action=
> based on information contained herein is strictly prohibited.
> Unauthorized use of information contained herein may subject you to > civil and criminal prosecution and penalties. If you are not the
> intended recipient, you should delete this message immediately and
> notify the sender immediately by telephone or by replying to this tran= smission.



--
Harsh J

This message and its contents (to include attachments) are the property of = National Health Systems, Inc. and may contain confidential and proprietary = information. This email and any files transmitted with it are intended sole= ly for the use of the individual or entity to whom they are addressed. You = are hereby notified that any unauthorized disclosure, copying, or distribut= ion of this message, or the taking of any unauthorized action based on info= rmation contained herein is strictly prohibited. Unauthorized use of inform= ation contained herein may subject you to civil and criminal prosecution an= d penalties. If you are not the intended recipient, you should delete this = message immediately and notify the sender immediately by telephone or by re= plying to this transmission.



--
= Alejandro

= --_000_B54B22AC36764A70B4D539D805538251yahooinccom_--