From issues-return-187801-archive-asf-public=cust-asf.ponee.io@spark.apache.org Sun Mar 25 12:36:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5A20418063B for ; Sun, 25 Mar 2018 12:36:05 +0200 (CEST) Received: (qmail 44251 invoked by uid 500); 25 Mar 2018 10:36:04 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 44242 invoked by uid 99); 25 Mar 2018 10:36:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 25 Mar 2018 10:36:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 04D011A086F for ; Sun, 25 Mar 2018 10:36:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -101.511 X-Spam-Level: X-Spam-Status: No, score=-101.511 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id XGqGy8jr-_JI for ; Sun, 25 Mar 2018 10:36:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 64F4B5FE08 for ; Sun, 25 Mar 2018 10:36:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6AE14E0147 for ; Sun, 25 Mar 2018 10:36:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 185202140B for ; Sun, 25 Mar 2018 10:36:00 +0000 (UTC) Date: Sun, 25 Mar 2018 10:36:00 +0000 (UTC) From: "Stavros Kontopoulos (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SPARK-23790?page=3Dcom.atlassia= n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D164= 12974#comment-16412974 ]=20 Stavros Kontopoulos commented on SPARK-23790: --------------------------------------------- [~q79969786] I see the PRs you created to fix the other PR, btw the doAsRea= lUser does the work: =C2=A0 {quote}18/03/23 19:26:18 DEBUG UserGroupInformation: PrivilegedAction as:hi= ve@LOCAL (auth:KERBEROS) from:org.apache.hadoop.hive.thrift.client.TUGIAssu= mingTransport.open(TUGIAssumingTransport.java:49) 18/03/23 19:26:18 DEBUG TSaslTransport: opening transport org.apache.thrift= .transport.TSaslClientTransport@64201482 18/03/23 19:26:18 DEBUG TSaslClientTransport: Sending mechanism name GSSAPI= and initial response of length 607 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Writing message with status= START and payload length 6 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Writing message with status= OK and payload length 607 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Start message handled 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Received message with statu= s OK and payload length 108 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Writing message with status= OK and payload length 0 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Received message with statu= s OK and payload length 32 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Writing message with status= COMPLETE and payload length 32 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Main negotiation loop compl= ete 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: SASL Client receiving last = message 18/03/23 19:26:18 DEBUG TSaslTransport: CLIENT: Received message with statu= s COMPLETE and payload length 0 18/03/23 19:26:18 INFO metastore: Connected to metastore. {quote} The reason is that I use an earlier branch to build stuff for the customer = which does not contain the commit. Thank you though there is a regression I= should know for the next releases and will follow the work being done. My = problem is that I tried to fetch delegation tokens earlier so consequent op= erations dont use a TGT all the time but hit this issue with HadoopRDD. I b= elieved I could add the delegation tokens when the mesos scheduler backend = starts like in the case of yarn where Client.java does something similar. > proxy-user failed connecting to a kerberos configured metastore > --------------------------------------------------------------- > > Key: SPARK-23790 > URL: https://issues.apache.org/jira/browse/SPARK-23790 > Project: Spark > Issue Type: Bug > Components: Mesos > Affects Versions: 2.3.0 > Reporter: Stavros Kontopoulos > Priority: Major > > This appeared at a customer trying to integrate with a kerberized hdfs cl= uster. > This can be easily fixed with the proposed fix [here|https://github.com/a= pache/spark/pull/17333]=C2=A0and the problem was reported first [here|https= ://issues.apache.org/jira/browse/SPARK-19995] for yarn. > The other option is to add the delegation tokens to the current user's UG= I as in [here|https://github.com/apache/spark/pull/17335] . The last fixes = the problem but leads to a failure when someones uses a HadoopRDD because t= he latter, uses FileInputFormat to get the splits which calls the local tic= ket cache by using TokenCache.obtainTokensForNamenodes. Eventually this wil= l fail with: > {quote}Exception in thread "main" org.apache.hadoop.ipc.RemoteException(j= ava.io.IOException): Delegation Token can be issued only with kerberos or w= eb authenticationat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.get= DelegationToken(FSNamesystem.java:5896) > {quote} > This implies that security mode is SIMPLE and hadoop libs there are not a= ware of kerberos. > This is related to this issue the workaround decided was=C2=A0to [trick|h= ttps://github.com/apache/spark/blob/a33655348c4066d9c1d8ad2055aadfbc892ba7f= d/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L795-L804] = hadoop. > =C2=A0 -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org