ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Veselovsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-2525) Yarn - Kerberos delegation token support for long running applications
Date Fri, 12 Feb 2016 11:18:18 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144461#comment-15144461
] 

Ivan Veselovsky commented on IGNITE-2525:
-----------------------------------------

Hi, Luca, 
I tried to reproduce the issue you describe (single-node kerberized HDP used), and the only
problem I see is an exception in the node manager log indicating that Log Aggregation subsystem
has failed to initialize (please see below). At the same time the yarn app itself works without
errors, the Ignite yarn application is able to run Ignite node(s), and the app logs are visible
through the Yarn web console, as the doc page prescribes. It would be nice to clarify the
following questions:
1) Is the exception you initially reported the same as below?
2) Does any other yarn or mapreduce application (e.g. hadoop wordcount example) run okay in
your kerberized cluster?
3) Is the problem you mention specific to long running applications only (that is, related
to ticket renewal problem only), or it prevents Ignite yarn module from running at all?

{code}
Caused by: java.io.IOException: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException:
Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "sandbox.hortonworks.com/10.0.2.15";
destination host is: "sandbox.hortonworks.com":8020;
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:773)
        at org.apache.hadoop.ipc.Client.call(Client.java:1431)
        at org.apache.hadoop.ipc.Client.call(Client.java:1358)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
        at com.sun.proxy.$Proxy13.getFileInfo(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
        at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
        at com.sun.proxy.$Proxy14.getFileInfo(Unknown Source)
        at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116)
        at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
        at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.checkExists(LogAggregationService.java:248)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.access$100(LogAggregationService.java:67)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService$1.run(LogAggregationService.java:276)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.createAppDir(LogAggregationService.java:261)
        ... 7 more

{code}

> Yarn - Kerberos delegation token support for long running applications
> ----------------------------------------------------------------------
>
>                 Key: IGNITE-2525
>                 URL: https://issues.apache.org/jira/browse/IGNITE-2525
>             Project: Ignite
>          Issue Type: Improvement
>    Affects Versions: 1.5.0.final
>            Reporter: Luca Rea
>            Assignee: Ivan Veselovsky
>             Fix For: 1.6
>
>
> Hi,
> now Yarn returns an authentication error when kerberos is enabled:
> java.io.IOException: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException:
Client cannot authenticate via:[TOKEN, KERBEROS]; 
> I think Apache Ignite needs somethink like "AMDelegationTokenRenewer.scala" and "ExecutorDelegationTokenUpdater.scala"
of Apache Spark to support long running applications in clusters with kerberos authantication
enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message