sentry-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vivek Shrivastava <vivshrivast...@gmail.com>
Subject Re: Unable to run latest Sentry build with Hive 1.1.0
Date Tue, 10 Mar 2015 14:07:34 GMT
For future readers. The exception has been resolved too. It was using an id
not authorized in the users allowed to connect. I have successfully
integrated latest hive with latest sentry trunk code.

On Tue, Mar 10, 2015 at 6:03 AM, Vivek Shrivastava <vivshrivastava@gmail.com
> wrote:

> Hi Xiaomeng,
>
> I noticed that the key was available in the hive trunk as well as apache
> hive 1.1.0 release. I realized that the hiveserver2 was taking the old hive
> 13.1 jars and causing the problem. I am building 1.5.0-incubating-SNAPSHOT on
> my own and built it successfully to and trying to integrate with latest
> Apache Hive. As Prasad has mentioned to me yesterday, latest release of
> Sentry and Apache hive should work.
>
> If you could help, I have a few questions:
>
> 1. Can I configure any Apache Hive release like 13.1 or latest hive trunk
> to use the old file based sentry-provider.ini ( I think it was the way in
> Sentry 1.2 ) instead of using as Latest hive 1.5 service?
>
> 2. I see that metastore message
>
> 15/03/10 09:32:33 WARN conf.HiveConf: HiveConf of name
> hive.sentry.conf.url does not exist
>
> 15/03/10 09:32:33 WARN conf.HiveConf: HiveConf of name hive.server2.enable.
> impersonation does not exist
>
> The properties are there in hive-site.xml then why is this message coming?
>
>
> 3. This is the content of sentry related configuration is hive-site.xml
>
> <property> <name>hive.server2.enable.impersonation</name> <description>Enable
> user impersonation for HiveServer2</description> <value>true</value>
> </property><property> <name> sentry.hive.provider.backend </name>
<value>
> org.apache.sentry.provider.file.SimpleFileProviderBackend </value>
> <description> The privilege provider to be used
>
> (either file-based or db-based). </description> </property>
>
> <property> <name>hive.sentry.conf.url</name> <value>
> file:///etc/sentry/conf/sentry-site.xml</value> </property>
>
> <property> <name>hive.metastore.pre.event.listeners</name> <value>
> org.apache.sentry.binding.metastore.MetastoreAuthzBinding</value>
> </property>
>
> <property> <name>hive.metastore.event.listeners</name> <value>
> org.apache.sentry.binding.metastore.SentryMetastorePostEventListener
> </value> </property>
>
> <property> <name>hive.server2.session.hook</name> <value>
> org.apache.sentry.binding.hive.HiveAuthzBindingSessionHook</value>
> </property>
>
> <property> <name>hive.security.authorization.task.factory</name> <value>
> org.apache.sentry.binding.hive.SentryHiveAuthorizationTaskFactoryImpl
> </value> </property>
>
> <property> <name>hive.server2.enable.doAs</name> <value>false</value>
> </property>
>
>
> 4. I am getting this error while running "show databases" or any command
> in beeline.
>
>
> 2015-03-10 09:59:16,173 WARN  [HiveServer2-Handler-Pool: Thread-21]:
> conf.HiveAuthzConf (HiveAuthzConf.java:get(206)) - Using the deprecated
> config setting hive.sentry.server instead of sentry.hive.server
>
> 2015-03-10 09:59:16,174 WARN  [HiveServer2-Handler-Pool: Thread-21]:
> conf.HiveAuthzConf (HiveAuthzConf.java:get(206)) - Using the deprecated
> config setting hive.sentry.provider instead of sentry.provider
>
> 2015-03-10 09:59:16,229 ERROR [HiveServer2-Handler-Pool: Thread-21]:
> ql.Driver (SessionState.java:printError(861)) - FAILED:
> InvocationTargetException null
>
> java.lang.reflect.InvocationTargetException
>
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>
>         at
> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.getAuthProvider(HiveAuthzBinding.java:205)
>
>         at
> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.<init>(HiveAuthzBinding.java:87)
>
>         at
> org.apache.sentry.binding.hive.authz.HiveAuthzBinding.<init>(HiveAuthzBinding.java:79)
>
>         at
> org.apache.sentry.binding.hive.HiveAuthzBindingHook.<init>(HiveAuthzBindingHook.java:97)
>
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>
>         at java.lang.Class.newInstance(Class.java:379)
>
>         at
> org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:60)
>
>         at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1297)
>
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:407)
>
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307)
>
>         at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112)
>
>         at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1106)
>
>         at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:101)
>
>         at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:172)
>
>         at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:379
> )
>
>         at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:366)
>
>         at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:271)
>
>         at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:415)
>
>         at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
>
>         at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
>
>         at
> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>
>         at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
>
>         at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
>
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
>         at java.lang.Thread.run(Thread.java:745)
>
> Caused by: java.lang.reflect.UndeclaredThrowableException
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1563)
>
>         at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl$UgiSaslClientTransport.open(SentryPolicyServiceClientDefaultImpl.java:104)
>
>         at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.<init>(SentryPolicyServiceClientDefaultImpl.java:156)
>
>         at
> org.apache.sentry.service.thrift.SentryServiceClientFactory.create(SentryServiceClientFactory.java:42)
>
>         at
> org.apache.sentry.provider.db.SimpleDBProviderBackend.<init>(SimpleDBProviderBackend.java:53)
>
>         at
> org.apache.sentry.provider.db.SimpleDBProviderBackend.<init>(SimpleDBProviderBackend.java:49)
>
>         ... 35 more
>
> Caused by: org.apache.thrift.transport.TTransportException: Peer indicated
> failure: Problem with callback handler
>
>         at
> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)
>
>         at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
>
>         at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>
>         at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl$UgiSaslClientTransport.baseOpen(SentryPolicyServiceClientDefaultImpl.java:120)
>
>         at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl$UgiSaslClientTransport.access$000(SentryPolicyServiceClientDefaultImpl.java:79)
>
>         at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl$UgiSaslClientTransport$1.run(SentryPolicyServiceClientDefaultImpl.java:106)
>
>         at
> org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl$UgiSaslClientTransport$1.run(SentryPolicyServiceClientDefaultImpl.java:104)
>
>         at java.security.AccessController.doPrivileged(Native Method)
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
>
>         ... 40 more
>
>
> 2015-03-10 09:59:16,229 INFO  [HiveServer2-Handler-Pool: Thread-21]:
> log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=compile
> start=1425981555715 end=1425981556229 duration=514
> from=org.apache.hadoop.hive.ql.Driver>
>
> 2015-03-10 09:59:16,229 INFO  [HiveServer2-Handler-Pool: Thread-21]:
> log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG
> method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
>
>
>
>
> On Tue, Mar 10, 2015 at 3:02 AM, Huang, Xiaomeng <xiaomeng.huang@intel.com
> > wrote:
>
>> Hi Vivek,
>> Did you set value "
>> org.apache.sentry.binding.hive.SentryHiveAuthorizationTaskFactoryImpl " for
>> the key "hive.security.authorization.task.factory" in hive conf?
>> If not, you should set this conf to use sentry task.
>> And what’s your version of sentry?
>> As far as I know, there is no release version of sentry to support apache
>> hive.
>> Current code base 1.5.0-incubating-SNAPSHOT in trunk could support apache
>> hive, but you may need to build it by yourself.
>>
>> Thanks,
>> Xiaomeng
>>
>>
>> -----Original Message-----
>> From: Vivek Shrivastava [mailto:vivshrivastava@gmail.com]
>> Sent: Tuesday, March 10, 2015 2:52 PM
>> To: dev
>> Subject: Re: Unable to run latest Sentry build with Hive 1.1.0
>>
>> Ah.. never mind. I see this is defined in the hive trunk.
>>
>> On Tue, Mar 10, 2015 at 2:33 AM, Vivek Shrivastava <
>> vivshrivastava@gmail.com
>> > wrote:
>>
>> > Hi,
>> >
>> > When I run the beeline command I get this error message. Do I have
>> > pass the value through command line? if so then what is the value?
>> >
>> > Thanks,
>> >
>> > Vivek
>> >
>> > 2015-03-10 06:18:30,157 ERROR [pool-7-thread-1]:
>> > thrift.ProcessFunction
>> > (ProcessFunction.java:process(41)) - Internal error processing
>> > OpenSession
>> >
>> > java.lang.NoSuchFieldError: HIVE_AUTHORIZATION_TASK_FACTORY
>> >
>> >         at
>> > org.apache.sentry.binding.hive.HiveAuthzBindingSessionHook.<clinit>(Hi
>> > veAuthzBindingSessionHook.java:46)
>> >
>> >         at java.lang.Class.forName0(Native Method)
>> >
>> >         at java.lang.Class.forName(Class.java:274)
>> >
>> >         at
>> > org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59)
>> >
>> >         at
>> > org.apache.hive.service.cli.session.SessionManager.executeSessionHooks
>> > (SessionManager.java:223)
>> >
>> >         at
>> > org.apache.hive.service.cli.session.SessionManager.openSession(Session
>> > Manager.java:136)
>> >
>> >         at
>> > org.apache.hive.service.cli.CLIService.openSession(CLIService.java:153
>> > )
>> >
>> >         at
>> > org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(T
>> > hriftCLIService.java:268)
>> >
>> >         at
>> > org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(Thrift
>> > CLIService.java:191)
>> >
>> >         at
>> > org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.g
>> > etResult(TCLIService.java:1253)
>> >
>> >         at
>> > org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.g
>> > etResult(TCLIService.java:1238)
>> >
>> >         at
>> > org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>> >
>> >         at
>> > org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> >
>> >         at
>> > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAss
>> > umingProcessor.process(HadoopThriftAuthBridge20S.java:628)
>> >
>> >         at
>> > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPo
>> > olServer.java:206)
>> >
>> >         at
>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j
>> > ava:1145)
>> >
>> >         at
>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
>> > java:615)
>> >
>> >         at java.lang.Thread.run(Thread.java:745)
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message