nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <mohit.j...@open-insights.co.in>
Subject RE: Unable to create HiveConnectionPool with kerberos.
Date Tue, 27 Mar 2018 09:59:53 GMT
Thanks Pierre, 

It is working now.

 

Mohit

 

From: Pierre Villard <pierre.villard.fr@gmail.com> 
Sent: 27 March 2018 13:15
To: users@nifi.apache.org
Subject: Re: Unable to create HiveConnectionPool with kerberos.

 

Hi,

It needs to be the principal of the Hive server, not yours. Can you give it a try by replacing
_HOST by the fqdn of your Hive server (just to check if that's the issue here)?

If you still have an error, I'd recommend checking the nifi-app.log file to have more detailed
(complete stack trace) about the "GSS initiate failed" error.

Pierre

 

2018-03-27 8:57 GMT+02:00 <mohit.jain@open-insights.co.in <mailto:mohit.jain@open-insights.co.in>
>:

When I try using my user prinicipal instead of hive it gives following error:

 

SelectHiveQL[id=633d54ed-0162-1000-0000-00006fa47d56] org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$523/1312347477@1c34a7fa
<mailto:org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$523/1312347477@1c34a7fa>
 failed to process due to java.lang.IllegalArgumentException: Kerberos principal should have
3 parts: mohit@olympus.oi.co.in <mailto:mohit@olympus.oi.co.in> ; rolling back session:
Kerberos principal should have 3 parts: mohit@olympus.oi.co.in <mailto:mohit@olympus.oi.co.in>


 

FYI…I am able to write to HDFS using kerberos. Just when I’m trying to create table in
hive using PutHiveQl, it throws the error.

 

 

From: mohit.jain@open-insights.co.in <mailto:mohit.jain@open-insights.co.in>  <mohit.jain@open-insights.co.in
<mailto:mohit.jain@open-insights.co.in> > 
Sent: 27 March 2018 11:22
To: users@nifi.apache.org <mailto:users@nifi.apache.org> 
Subject: RE: Unable to create HiveConnectionPool with kerberos.

 

Hi,

 

I have tried that URL but it gives me following error:-

 

HiveConnectionPool[id=6e60258b-9e00-3bac-9590-543aec882280] Error getting Hive connection:
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could
not open client transport with JDBC Uri: jdbc:hive2://**.co.in:10000/nifi_test1;principal=hive/_HOST@**.co.in
<http://co.in:10000/nifi_test1;principal=hive/_HOST@**.co.in> : GSS initiate failed)

 

 

Mohit

 

 

From: Pierre Villard <pierre.villard.fr@gmail.com <mailto:pierre.villard.fr@gmail.com>
> 
Sent: 26 March 2018 23:31
To: users@nifi.apache.org <mailto:users@nifi.apache.org> 
Subject: Re: Unable to create HiveConnectionPool with kerberos.

 

Mohit,

I believe you need to change the JDBC url (even though you have the configuration files correctly
set) to something like:
jdbc:hive2://<host>:<port>/<db>;principal=<Server_Principal_of_HiveServer2>

So it'd be something like: 

jdbc:hive2://localhost:10000/default;principal=hive/my.fqdn.hive.server@EXAMPLE.COM <mailto:my.fqdn.hive.server@EXAMPLE.COM>



Pierre

 

2018-03-26 18:13 GMT+02:00 Juan Pablo Gardella <gardellajuanpablo@gmail.com <mailto:gardellajuanpablo@gmail.com>
>:

Sorry, the issue happens when a HA configuration is used.

 

On Mon, 26 Mar 2018 at 13:03 Juan Pablo Gardella <gardellajuanpablo@gmail.com <mailto:gardellajuanpablo@gmail.com>
> wrote:

See https://issues.apache.org/jira/browse/NIFI-2575, the driver does not suppor that. I've
put some workarounds in the ticket.

 

On Mon, 26 Mar 2018 at 13:03 <mohit.jain@open-insights.co.in <mailto:mohit.jain@open-insights.co.in>
> wrote:

Hi,

 

I am getting the following warning when I use HiveConnection pool with Kerberos :

 

HiveConnectionPool[id=6e60258b-9e00-3bac-85ba-0dac8e22142f] Configuration does not have security
enabled, Keytab and Principal will be ignored

 

It also throws the following bulletin in my PutHiveQl processor: 

PutHiveQL[id=55f4ac1b-ecf9-3db3-b898-7a9d145a5382] org.apache.nifi.processors.hive.PutHiveQL$$Lambda$663/2042832677
<tel:(204)%20283-2677> @40267000 failed to process due to org.apache.nifi.processor.exception.ProcessException:
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could
not open client transport with JDBC Uri: jdbc:hive2://******:10000/nifi_test1: Peer indicated
failure: Unsupported mechanism type PLAIN); rolling back session: org.apache.commons.dbcp.SQLNestedException:
Cannot create PoolableConnectionFactory (Could not open client transport with JDBC Uri: jdbc:hive2://******:10000/nifi_test1:
Peer indicated failure: Unsupported mechanism type PLAIN)
 

Hive Configuration Resources:- /etc/hive/conf/hive-site.xml,/etc/hadoop/conf/core-site.xml

I have set hive.security.authentication and hadoop.security.authentication to Kerberos.

 

Please let me know if I’m doing anything wrong.

 

Regards,

Mohit

 

 


Mime
View raw message