hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yongqiang he <heyongqiang...@gmail.com>
Subject Re: Hive 0.7.1 authorization woes
Date Fri, 26 Aug 2011 00:22:12 GMT
what is your unix name on that machine? can u do a whoami?

On Thu, Aug 25, 2011 at 5:15 PM, Alex Holmes <grep.alex@gmail.com> wrote:
> Here's the hive-site.xml file (I use the same file for both the client
> and remote metastore).  We're using mysql as the metastore DB.
>
>
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <configuration>
> <property>
>  <name>hive.security.authorization.enabled</name>
>  <value>true</value>
> </property>
> <property>
>  <name>hive.metastore.local</name>
>  <value>false</value>
> </property>
> <property>
>  <name>hive.metastore.uris</name>
>  <value>thrift://localhost:9083</value>
> </property>
> <property>
>  <name>javax.jdo.option.ConnectionURL</name>
>  <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
> </property>
> <property>
>  <name>javax.jdo.option.ConnectionDriverName</name>
>  <value>com.mysql.jdbc.Driver</value>
> </property>
> <property>
>  <name>javax.jdo.option.ConnectionUserName</name>
>  <value>hive</value>
> </property>
> <property>
>  <name>javax.jdo.option.ConnectionPassword</name>
>  <value>secret</value>
> </property>
> </configuration>
>
>
>
> On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he <heyongqiangict@gmail.com> wrote:
>> this is what i have tried with a remote metastore:
>>
>>    > set hive.security.authorization.enabled=false;
>> hive>
>>    >
>>    >
>>    > drop table src2;
>> OK
>> Time taken: 1.002 seconds
>> hive> create table src2 (key int, value string);
>> OK
>> Time taken: 0.03 seconds
>> hive>
>>    >
>>    >
>>    > set hive.security.authorization.enabled=true;
>> hive> grant select on table src2 to user heyongqiang;
>> OK
>> Time taken: 0.113 seconds
>> hive> select * from src2;
>> OK
>> Time taken: 0.188 seconds
>> hive> show grant user heyongqiang on table src2;
>> OK
>>
>> database        default
>> table   src2
>> principalName   heyongqiang
>> principalType   USER
>> privilege       Select
>> grantTime       Wed Aug 24 15:03:51 PDT 2011
>> grantor heyongqiang
>>
>> can u do a show grant?
>>
>> (But with remote metastore, i think hive should not return empty list
>> instead of null for list_privileges etc.)
>>
>>
>>
>> On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <grep.alex@gmail.com> wrote:
>>> Authorization works for me with the local metastore.  The remote
>>> metastore works with authorization turned off, but as soon as I turn
>>> it on and issue any commands I get these exceptions on the hive
>>> client.
>>>
>>> Could you also try the remote metastore please?  I'm pretty sure that
>>> authorization does not work with it at all.
>>>
>>> Thanks,
>>> Alex
>>>
>>> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <heyongqiangict@gmail.com>
wrote:
>>>> I am using local metastore,  and can not reproduce the problem.
>>>>
>>>> what message did you get when running local metastore?
>>>>
>>>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <grep.alex@gmail.com>
wrote:
>>>>> Thanks for opening a ticket.
>>>>>
>>>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>>>> that the bug is only related to global grants).
>>>>>
>>>>> hive> set hive.security.authorization.enabled=false;
>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>> OK
>>>>> Time taken: 1.245 seconds
>>>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO
TABLE pokes;
>>>>> Copying data from file:/app/hadoop/hive1.in
>>>>> Copying file: file:/app/hadoop/hive1.in
>>>>> Loading data to table default.pokes
>>>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>>>> OK
>>>>> Time taken: 0.33 seconds
>>>>> hive> select * from pokes;
>>>>> OK
>>>>> 1       a
>>>>> 2       b
>>>>> 3       c
>>>>> Time taken: 0.095 seconds
>>>>> hive> grant select on table pokes to user hduser;
>>>>> OK
>>>>> Time taken: 0.251 seconds
>>>>> hive> set hive.security.authorization.enabled=true;
>>>>> hive> select * from pokes;
>>>>> FAILED: Hive Internal Error:
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>> get_privilege_set failed: unknown result)
>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>> unknown result
>>>>>        at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>> ...
>>>>>
>>>>> mysql> select * from TBL_PRIVS;
>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
|
>>>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>> |            1 |  1314219701 |            0 | hduser  |
USER         |
>>>>> hduser         | USER           | Select   |      1 |
>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>
>>>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>>>
>>>>>  Authorization failed:No privilege 'Create' found for outputs {
>>>>> database:default}. Use show grant to get more details.
>>>>>
>>>>> Whereas I just get an exception (as you can see above).  Were you also
>>>>> running with the remote metastore?  I get these meaningful messages
>>>>> with the local metastore (and authorization on), but with the remote
>>>>> metastore with authorization turned on, I always get exceptions.
>>>>>
>>>>> Many thanks,
>>>>> Alex
>>>>>
>>>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <heyongqiangict@gmail.com>
wrote:
>>>>>> This is a bug. Will open a jira to fix this. and will backport it
to 0.7.1.
>>>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>>>
>>>>>> thanks for reporting this one!
>>>>>>
>>>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <grep.alex@gmail.com>
wrote:
>>>>>>> I created the mysql database (with the simple create database
command)
>>>>>>> and the remote metastore seemed to creat the mysql tables.  Here's
>>>>>>> some grant information and what I see in the database:
>>>>>>>
>>>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>>>> hive> grant all to user hduser;
>>>>>>> OK
>>>>>>> Time taken: 0.334 seconds
>>>>>>> hive> show grant user hduser;
>>>>>>> OK
>>>>>>>
>>>>>>> principalName   hduser
>>>>>>> principalType   USER
>>>>>>> privilege       All
>>>>>>> grantTime       1314191500
>>>>>>> grantor hduser
>>>>>>> Time taken: 0.046 seconds
>>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>>> FAILED: Hive Internal Error:
>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>> get_privilege_set failed: unknown result)
>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>> unknown result
>>>>>>>        at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>> ...
>>>>>>>
>>>>>>> mysql> use hive;
>>>>>>> Database changed
>>>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>> |             1 |  1314191500 |            0 | hduser
 | USER
>>>>>>> | hduser         | USER           | All       |
>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>> 1 row in set (0.00 sec)
>>>>>>>
>>>>>>>
>>>>>>> Thanks for your help,
>>>>>>> Alex
>>>>>>>
>>>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he <heyongqiangict@gmail.com>
wrote:
>>>>>>>> Have you created the metastore mysql tables for authorization?
Can u
>>>>>>>> do a show grant?
>>>>>>>>
>>>>>>>> thanks
>>>>>>>> yongqiang
>>>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <grep.alex@gmail.com>
wrote:
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I've been struggling with getting Hive authorization
to work for a few
>>>>>>>>> hours, and I really hope someone can help me.  I installed
Hive 0.7.1
>>>>>>>>> on top of Hadoop 0.20.203.  I'm using mysql for the
metastore, and
>>>>>>>>> configured Hive to enable authorization:
>>>>>>>>>
>>>>>>>>> <property>
>>>>>>>>>  <name>hive.security.authorization.enabled</name>
>>>>>>>>>  <value>true</value>
>>>>>>>>>  <description>enable or disable the hive client
authorization</description>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>> I kept all the other Hive security configs with their
default settings.
>>>>>>>>>
>>>>>>>>> I'm running in pseudo-distributed mode on a single node.
 HDFS, the Hive
>>>>>>>>> metastore and the Hive CLI are all running as the same
user (the HDFS
>>>>>>>>> superuser).  Here are the sequence of steps that are
causing me issues.
>>>>>>>>> Without authorization everything works perfectly (creating,
loading, selecting).
>>>>>>>>> I've also tried creating and loading the table without
authorization, granting
>>>>>>>>> the select privilege at various levels (global, table,
database), turning on
>>>>>>>>> auth and performing the select, resulting in the same
exception.
>>>>>>>>>
>>>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Alex
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>>>> Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>>>> hive> grant all to user hduser;
>>>>>>>>> OK
>>>>>>>>> Time taken: 0.233 seconds
>>>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set
failed:
>>>>>>>>> unknown result
>>>>>>>>>        at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>>>        at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>>>        at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>>>>        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>>>>        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>>>>        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>>>>        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>>>>        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>>>>>>>>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>>>> failed: unknown result
>>>>>>>>>        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>>>>        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>>>>        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>>>>        at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>>>>        ... 14 more
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message