hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath" <prade...@yahoo-inc.com>
Subject RE: [howldev] Initial thoughts on authorization in howl
Date Fri, 30 Jul 2010 00:10:48 GMT
Hi Ashish,
   The changes I was mentioning in the mail were changes in hive code.
For howl, we will have howl specific semantic analyzers which will
enforce authorization for ddl (like create table/drop table) against
hdfs permissions. This is our initial thought on authorization for DDL
through howl CLI - note, this does NOT change anything for hive CLI. I
did notice there was jira in hive for authorization which seems similar
to SQL authorization. The big issue there is reconciling SQL permissions
with hdfs permissions
(https://issues.apache.org/jira/browse/HIVE-78?focusedCommentId=12682719
&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel
#action_12682719)

So for now, we are going the hdfs route initially in howl - this may not
be the final story for authorization in howl but is a beginning.

I was initially thinking of using the hive conf variables and not table
properties but I guess either could be used. Initially I thought we
won't need to persist the group and permissions in the metastore and
just use that info while creating the table dir. Later when creating
partition dirs, we would just consult the table dir. If we use table
properties, we can persist the group/perms info in the metastore and
partition creation can get it from the metastore.


Pradeep

-----Original Message-----
From: Ashish Thusoo [mailto:athusoo@facebook.com] 
Sent: Thursday, July 29, 2010 3:01 PM
To: hive-dev@hadoop.apache.org
Cc: howldev@yahoogroups.com
Subject: RE: [howldev] Initial thoughts on authorization in howl

Hi Pradeep,

I get from this note that the authorization that you are talking about
here are basically the management of the permissions on the hdfs
directories corresponding to the tables and the partitions. So from that
angle this sounds good to me. There is a whole set of
permissions/authorizations with regard to the metadata operations
themselves eg. Who should be able to run an alter table add column or
describe table etc. I presume that would be beyond the scope of this
change and would come in later? I am thinking more in terms of the
permissions model that is supported in SQL using GRANT statements etc.

I also presume that by conf variables you mean the key value properties
that Hive can store in the metadata and not the hive conf variables,
right?

Ashish

-----Original Message-----
From: John Sichi [mailto:jsichi@facebook.com] 
Sent: Wednesday, July 28, 2010 2:22 PM
To: hive-dev@hadoop.apache.org
Subject: Fwd: [howldev] Initial thoughts on authorization in howl

Begin forwarded message:

From: Pradeep Kamath
<pradeepk@yahoo-inc.com<mailto:pradeepk@yahoo-inc.com>>
Date: July 27, 2010 4:38:42 PM PDT
To: <howldev@yahoogroups.com<mailto:howldev@yahoogroups.com>>
Subject: [howldev] Initial thoughts on authorization in howl
Reply-To: <howldev@yahoogroups.com<mailto:howldev@yahoogroups.com>>



The initial thoughts on authorization in howl are to model authorization
(for DDL ops like create table/drop table/add partition etc) after hdfs
permissions. To be able to do this, we would like to extend
createTable() to add the ability to record a different group from the
user's primary group and to record the complete unix permissions on the
table directory. Also, we would like to have a way for partition
directories to inherit permissions and group information based on the
table directory. To keep the metastore backward compatible for use with
hive, I propose having conf variables to achieve these objectives:
-          table.group.name<http://table.group.name> - value will
indicate the name of the unix group for the table directory. This will
be used by createTable() to perform a chgrp to the value provided. This
property will provide the user the ability to choose from one of the
many unix groups he is part of to associate with the table.
-          table.permissions - value will be of the form rwxrwxrwx to
indicate read-write-execute permissions on the table directory. This
will be used by createTable() to perform a chmod to the value provided.
This will let the user decide what permissions he wants on the table.
-          partitions.inherit.permissions - a value of true will
indicate that partitions inherit the group name and permissions of the
table level directory. This will be used by addPartition() to perform a
chgrp and chmod to the values as on the table directory.

I favor conf properties over API changes since the complete
authorization design for hive is not finalized yet. These properties can
be deprecated/removed when that is in place. These properties would also
be useful to some installation of vanilla hive since at least DFS level
authorization can now be achieved by hive without the user having to
manually perform chgrp and chmod operations on DFS.

I would like to hear from hive developers/committers whether this would
be acceptable for hive and also thoughts from others.

Pradeep



__._,_.___


Your email settings: Individual Email|Traditional Change settings via
the
Web<http://groups.yahoo.com/group/howldev/join;_ylc=X3oDMTJnZXE5ZHNwBF9T
Azk3NDc2NTkwBGdycElkAzYzNDIwNTA4BGdycHNwSWQDMTcwNzI4MTk0MgRzZWMDZnRyBHNs
awNzdG5ncwRzdGltZQMxMjgwMjczOTQ2> (Yahoo! ID required) Change settings
via email: Switch delivery to Daily
Digest<mailto:howldev-digest@yahoogroups.com?subject=Email%20Delivery:%2
0Digest> | Switch to Fully
Featured<mailto:howldev-fullfeatured@yahoogroups.com?subject=Change%20De
livery%20Format:%20Fully%20Featured>
Visit Your Group
<http://groups.yahoo.com/group/howldev;_ylc=X3oDMTJlOWw0Y3F0BF9TAzk3NDc2
NTkwBGdycElkAzYzNDIwNTA4BGdycHNwSWQDMTcwNzI4MTk0MgRzZWMDZnRyBHNsawNocGYE
c3RpbWUDMTI4MDI3Mzk0Ng--> | Yahoo! Groups Terms of Use
<http://docs.yahoo.com/info/terms/> | Unsubscribe
<mailto:howldev-unsubscribe@yahoogroups.com?subject=Unsubscribe>

__,_._,___


Mime
View raw message