hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2746) [MR-279] [Security] Yarn servers can't communicate with each other with hadoop.security.authorization set to true
Date Sat, 22 Oct 2011 21:38:32 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133491#comment-13133491
] 

Mahadev konar commented on MAPREDUCE-2746:
------------------------------------------

Looks like on the first attempt of installing a cluster with this on, causes jobs to fail
but on reinstall the jobs start passing. Here is the stack trace:

{noformat}

11/10/22 21:17:11 INFO mapreduce.Job: Job job_1319318025575_0003 failed with state FAILED
due to: Application application_1319318025575_0003 failed 5 times due to Error launchi
ng appattempt_1319318025575_0003_000005. Got exception: java.lang.reflect.UndeclaredThrowableException
        at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:88)
       at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:120)
        at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:261)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:662)
Caused by: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception:
java.io.IOException: java.io.IOException: Failed to specify server's Kerberos principal name;
Host Details : local host is: "RMHOST"; destination host is: "NMHOST";
        at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:139)
        at $Proxy17.startContainer(Unknown Source)
        at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:81)
       ... 5 more
Caused by: java.io.IOException: Failed on local exception: java.io.IOException: java.io.IOException:
Failed to specify server's Kerberos principal name; Host Details : local ho
st is: "RMHOST"; destination host is: "NMHOST";
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:634)
        at org.apache.hadoop.ipc.Client.call(Client.java:1089)
        at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:136)
        ... 7 more
Caused by: java.io.IOException: java.io.IOException: Failed to specify server's Kerberos principal
name
        at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:535)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
        at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:499)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:583)
        at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:205)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1195)
        at org.apache.hadoop.ipc.Client.call(Client.java:1065)
        ... 8 more
Caused by: java.io.IOException: Failed to specify server's Kerberos principal name
        at org.apache.hadoop.security.SaslRpcClient.<init>(SaslRpcClient.java:88)
        at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:406)
        at org.apache.hadoop.ipc.Client$Connection.access$1200(Client.java:205)
        at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:576)
        at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:573)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:572)
        ... 11 more
. Failing the application.

{noformat}
                
> [MR-279] [Security] Yarn servers can't communicate with each other with hadoop.security.authorization
set to true
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2746
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2746
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2, security
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2746.patch, MAPREDUCE-2746.patch, MAPREDUCE-2746.patch,
MAPREDUCE-2746.patch, MAPREDUCE-2746.patch
>
>
> Because of this problem, till now, we've been testing YARN+MR with {{hadoop.security.authorization}}
set to false. We need to register yarn communication protocols in the implementation of the
authorization related PolicyProvider (MapReducePolicyProvider.java).
> [~devaraj] also found this issue independently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message