hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3007) JobClient cannot talk to JobHistory server in secure mode
Date Thu, 15 Sep 2011 02:48:09 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105079#comment-13105079
] 

Mahadev konar commented on MAPREDUCE-3007:
------------------------------------------

Vinod,
 The conf property setting is not needed any longer:

{code}
     myConf.setClass(YarnConfiguration.YARN_SECURITY_INFO,
-        ClientRMSecurityInfo.class, SecurityInfo.class);
-    YarnRPC rpc = YarnRPC.create(myConf);
+        ClientHSSecurityInfo.class, SecurityInfo.class);
{code}

Setting the conf property is redundant since SecurityUtil picks it up via going through the
providers. The only issue with HS is that both the AM and HS implement the same interface
(MRClientProtocol) one of which is token based and the other kerberos info based (the fact
that getKerberos in MRClientSecurityInfo returns null saves the day). 


I tried out the patch on a cluster. Had to remove the sleep in AppMaster to reproduce the
issue. The patch worked for me, though I started getting some other error on getTaskCompletions().
I think thats more related to the handshake for history file transfer not getting completed
becasue of immediate AppMaster shutdown.

In all, the patch looks good with minor changes to removing the conf settings mentioned above.

> JobClient cannot talk to JobHistory server in secure mode
> ---------------------------------------------------------
>
>                 Key: MAPREDUCE-3007
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3007
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-3007-20110914.2.txt, MAPREDUCE-3007-20110914.txt
>
>
> In secure mode, Jobclient cannot connect to HistoryServer. Thanks to [~karams] for finding
this out.
> {code}
> 11/09/14 09:57:51 INFO mapred.ClientServiceDelegate: Application state is completed.
Redirecting to job history server
> 11/09/14 09:57:51 INFO security.ApplicationTokenSelector: Looking for a token with service
<history-server>:10020
> 11/09/14 09:57:51 INFO security.ApplicationTokenSelector: Token kind is YARN_APPLICATION_TOKEN
and the token's service name is <Am-ip>:46257
> 11/09/14 09:57:51 INFO security.UserGroupInformation: Initiating logout for <user-principal>
> 11/09/14 09:57:51 INFO security.UserGroupInformation: Initiating re-login for <user-principal>
> 11/09/14 09:57:55 WARN security.UserGroupInformation: Not attempting to re-login since
the last re-login was attempted less than 600 seconds before.
> 11/09/14 09:57:56 WARN security.UserGroupInformation: Not attempting to re-login since
the last re-login was attempted less than 600 seconds before.
> 11/09/14 09:58:00 WARN security.UserGroupInformation: Not attempting to re-login since
the last re-login was attempted less than 600 seconds before.
> 11/09/14 09:58:05 WARN security.UserGroupInformation: Not attempting to re-login since
the last re-login was attempted less than 600 seconds before.
> 11/09/14 09:58:05 WARN ipc.Client: Couldn't setup connection for <user-principal>
to null
> 11/09/14 09:58:05 INFO mapred.ClientServiceDelegate: Failed to contact AM/History for
job job_1315993268700_0001  Will retry..
> {code}
> Am surprised no one working with YARN+MR ever ran into this!

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message