nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Storck (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-3472) PutHDFS Kerberos relogin not working (tgt) after ticket expires
Date Mon, 07 Aug 2017 13:34:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116593#comment-16116593
] 

Jeff Storck commented on NIFI-3472:
-----------------------------------

[~jomach] The fix I'm working on for this JIRA will be applied to all of NiFi's HDFS processors,
and eventually the HBase and Hive processors. I'll update the JIRA to reflect this.  Thanks!

> PutHDFS Kerberos relogin not working (tgt) after ticket expires
> ---------------------------------------------------------------
>
>                 Key: NIFI-3472
>                 URL: https://issues.apache.org/jira/browse/NIFI-3472
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.0.0, 1.1.0, 1.1.1, 1.0.1
>            Reporter: Jeff Storck
>            Assignee: Jeff Storck
>
> PutHDFS is not able to relogin if the ticket expires.
> NiFi, running locally as standalone, was sending files to HDFS.  After suspending the
system for the weekend, when the flow attempted to continue to process flowfiles, the following
exception occurred:
> {code}2017-02-13 11:59:53,460 WARN [Timer-Driven Process Thread-10] org.apache.hadoop.ipc.Client
Exception encountered while connecting to the server : javax.security.sasl.SaslException:
GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos tgt)]
> 2017-02-13 11:59:53,463 INFO [Timer-Driven Process Thread-10] o.a.h.io.retry.RetryInvocationHandler
Exception while invoking getFileInfo of class ClientNamenodeProtocolTranslatorPB over [host:port]
after 3 fail over attempts. Trying to fail over immediately.
> java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException:
GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos tgt)]; Host Details : local host is: "[host:port]"; destination
host is: [host:port];
> 	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776) ~[hadoop-common-2.7.3.jar:na]
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1479) ~[hadoop-common-2.7.3.jar:na]
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1412) ~[hadoop-common-2.7.3.jar:na]
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
~[hadoop-common-2.7.3.jar:na]
> 	at com.sun.proxy.$Proxy136.getFileInfo(Unknown Source) ~[na:na]
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
~[hadoop-hdfs-2.7.3.jar:na]
> 	at sun.reflect.GeneratedMethodAccessor386.invoke(Unknown Source) ~[na:na]
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_102]
> 	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
~[hadoop-common-2.7.3.jar:na]
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
~[hadoop-common-2.7.3.jar:na]
> 	at com.sun.proxy.$Proxy137.getFileInfo(Unknown Source) [na:na]
> 	at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) [hadoop-hdfs-2.7.3.jar:na]
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
[hadoop-hdfs-2.7.3.jar:na]
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
[hadoop-hdfs-2.7.3.jar:na]
> 	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
[hadoop-common-2.7.3.jar:na]
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)
[hadoop-hdfs-2.7.3.jar:na]
> 	at org.apache.nifi.processors.hadoop.PutHDFS$1.run(PutHDFS.java:262) [nifi-hdfs-processors-1.1.1.jar:1.1.1]
> 	at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_102]
> 	at javax.security.auth.Subject.doAs(Subject.java:360) [na:1.8.0_102]
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1678)
[hadoop-common-2.7.3.jar:na]
> 	at org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:230) [nifi-hdfs-processors-1.1.1.jar:1.1.1]
> 	at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
[nifi-api-1.1.1.jar:1.1.1]
> 	at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
[nifi-framework-core-1.1.1.jar:1.1.1]
> 	at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.1.1.jar:1.1.1]
> 	at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.1.1.jar:1.1.1]
> 	at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.1.1.jar:1.1.1]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_102]
> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_102]
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_102]
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_102]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_102]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_102]
> 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]{code}
> After stopping and starting the PutHDFS processor, flowfiles were able to be transferred
to HDFS.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message