Return-Path: X-Original-To: apmail-nifi-commits-archive@minotaur.apache.org Delivered-To: apmail-nifi-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C00B418A31 for ; Wed, 11 Nov 2015 15:19:11 +0000 (UTC) Received: (qmail 98702 invoked by uid 500); 11 Nov 2015 15:19:11 -0000 Delivered-To: apmail-nifi-commits-archive@nifi.apache.org Received: (qmail 98666 invoked by uid 500); 11 Nov 2015 15:19:11 -0000 Mailing-List: contact commits-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list commits@nifi.apache.org Received: (qmail 98635 invoked by uid 99); 11 Nov 2015 15:19:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Nov 2015 15:19:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 4F1C12C1F5B for ; Wed, 11 Nov 2015 15:19:11 +0000 (UTC) Date: Wed, 11 Nov 2015 15:19:11 +0000 (UTC) From: "ASF subversion and git services (JIRA)" To: commits@nifi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (NIFI-1062) PutHDFS will pass files to success when they were not successfully written with hadoop client misconfiguration MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/NIFI-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000494#comment-15000494 ] ASF subversion and git services commented on NIFI-1062: ------------------------------------------------------- Commit 62e3cfc6291d1f30ac92f29cf790336644205919 in nifi's branch refs/heads/master from [~ozhurakousky] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=62e3cfc ] NIFI-1024 NIFI-1062 Fixed PutHDFS processor to properly route failures. Ensured that during put failures the FlowFile is routed to 'failure' relationship. Added validation test Re-enabled previously ignored test. Signed-off-by: Bryan Bende > PutHDFS will pass files to success when they were not successfully written with hadoop client misconfiguration > -------------------------------------------------------------------------------------------------------------- > > Key: NIFI-1062 > URL: https://issues.apache.org/jira/browse/NIFI-1062 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Affects Versions: 0.3.0 > Environment: hadoop-2.6.0 with hadoop.security.authentication=kerberos and dfs.data.transfer.protection=privacy > Reporter: Tony Kurc > Assignee: Oleg Zhurakousky > Labels: hdfs, kerberos > Fix For: 0.4.0 > > > PutHDFS will create an empty file, but the data it is attempting to write will fail with this stack trace, and the flow file gets routed to success. > {noformat} > 2015-10-24 11:16:19,278 WARN [Thread-4674] org.apache.hadoop.hdfs.DFSClient DataStreamer Exception > java.lang.IllegalArgumentException: null > at javax.security.auth.callback.NameCallback.(NameCallback.java:90) ~[na:1.8.0_45] > at com.sun.security.sasl.digest.DigestMD5Client.processChallenge(DigestMD5Client.java:324) ~[na:1.8.0_45] > at com.sun.security.sasl.digest.DigestMD5Client.evaluateChallenge(DigestMD5Client.java:220) ~[na:1.8.0_45] > at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslParticipant.evaluateChallengeOrResponse(SaslParticipant.java:113) ~[na:na] > at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:451) ~[na:na] > at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getSaslStreams(SaslDataTransferClient.java:390) ~[na:na] > at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:262) ~[na:na] > at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.checkTrustAndSend(SaslDataTransferClient.java:210) ~[na:na] > at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.socketSend(SaslDataTransferClient.java:182) ~[na:na] > at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1413) ~[na:na] > at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1361) ~[na:na] > at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588) ~[na:na] > 2015-10-24 11:16:19,303 INFO [Timer-Driven Process Thread-2] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=3c30a474-86b4-45fc-b771-95d7a1b5054d] copied StandardFlowFileRecord[uuid=5a3deada-9739-474f-a83d-0447ad5aefd9,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1445694733273-1, container=default, section=1], offset=87, length=29],offset=0,name=x11.txt,size=29] to HDFS at /use/hdfs/x11.txt in 56 milliseconds at a rate of 511 bytes/sec > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)