hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Subramanian <Sanjay.Subraman...@wizecommerce.com>
Subject Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?
Date Tue, 26 Mar 2013 17:51:22 GMT
Hi

I added the following to hive-site.xml

 <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
  </property>


I did not add hive.security.authorization.manager because I am currently using Cloudera Manager
4.1.2 to install all components and looks like they don't have a Hcatalog distribution that
I need for the hive.security.authorization.manager param to work correctly (after pointing
the HIVE_AUX_JARS to this Hcatalog jar location)

:-) I can do a tarball install on my sandbox if needed later but at this moment I am trying
to stabilize the steps for Cloudera CDH4.1.2 install so that I can tell the OPS guys to follow
the steps…they will find it difficult to manually install on N node clusters in production
and I think that also will be highly error prone


Anyway the add partitions works from the Hive CLI and Beeswax BUT fails when I use the HiveService
interface (This is a oozie workflow created in Hue and submitted to Hadoop Yarn Cluster)

HiveServerException(message:Query returned non-zero code: 403, cause: null, errorCode:403,
SQLState:null) at
org.apache.hadoop.hive.service.ThriftHive$execute_result.read(ThriftHive.java:1333) at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at
org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:106) at
org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:93) at
com.wizecommerce.utils.hive.service.HiveServiceBAction.runActionStr(HiveServiceBAction.java:46)
at
com.wizecommerce.utils.hive.service.OutpdirHeaderAddPartition.addPartitions(OutpdirHeaderAddPartition.java:105)
at
com.wizecommerce.utils.hive.service.OutpdirHeaderAddPartition.main(OutpdirHeaderAddPartition.java:69)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at
java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152) at java.security.AccessController.doPrivileged(Native
Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)

Thanks
sanjay


From: Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Monday, March 25, 2013 8:08 PM
To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?

if you can share the error when you add partitions after enabling the property that will give
more hints


On Tue, Mar 26, 2013 at 8:38 AM, Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
wrote:
Can you enable this property ?
also can you run "show grant user userid" ?

there was a bug for this https://issues.apache.org/jira/browse/HIVE-2405 fixed in 0.8.0 so
hopefully you will not hit this.

if enable this property and grant permissions to create table to the user, it means user can
add partitions.
from language manual
  *   CREATE - Allows users to create objects. For a database, this means users can create
tables, and for a table, this means users can create partitions .. you can refer the entire
table at https://cwiki.apache.org/Hive/languagemanual-auth.html



On Tue, Mar 26, 2013 at 8:24 AM, Sanjay Subramanian <Sanjay.Subramanian@wizecommerce.com<mailto:Sanjay.Subramanian@wizecommerce.com>>
wrote:
I am using
Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2
My metastore is MySQL

My hive.security.authorization.enabled is set to false as of now…I am not able to add partitions
if I keep that as true

 <property>
    <name>hive.security.authorization.enabled</name>
    <value>false</value>
    <description>enable or disable the hive client authorization</description>
  </property>


I have not defined a property=hive.security.authorization.manager

Thanks
sanjay

From: Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Monday, March 25, 2013 7:43 PM

To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?

YARN should not play any role in any create table statement. It just creates a directory with
DFSClient. Normally it tries to create it with the cli userid in my experience.

which version of hive are you using?
which is your metastore?

Can you check for the following values?

 <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
  </property>

  <property>
    <name>hive.security.authorization.manager</name>
    <value>org.apache.hcatalog.security.HdfsAuthorizationProvider</value>
    <description>the hive client authorization manager class name.
    The user defined authorization class should implement interface
    org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
    </description>
  </property>


On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian <Sanjay.Subramanian@wizecommerce.com<mailto:Sanjay.Subramanian@wizecommerce.com>>
wrote:
Hi Nitin
I notice this peculiarity in Yarn and Hive
I have another earlier cluster with MRv1 where I have created and run several hive tables
and scripts ; The same test Create Table script gives the correct owner name
I added location but that did not help
hive -e "CREATE TABLE name (id INT,  name STRING) LOCATION '/user/hive/warehouse/name';"

Thanks
Sanjay

From: Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Monday, March 25, 2013 7:13 PM
To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ?

Forgot to add,
if you want full filesystem level security on HDFS then you will need to enable kerberos based
security.


On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>>
wrote:
Sanjay,

can you try adding 'LOCATION' clause to your create statement.
By default the hive warehouse directory is writable by all the user. To create it by the individual
users you need to provide by the location clause.


On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian <Sanjay.Subramanian@wizecommerce.com<mailto:Sanjay.Subramanian@wizecommerce.com>>
wrote:
Steps to recreate the use case:

- Log in as sasubramanian to Linux Box
- Execute hive -e "CREATE TABLE name (id INT,  name STRING);"
- Go to HDFS /user/hive/warehouse/

Name  Type  Size  Replication  Block Size  Modification Time      Permission  Owner  Group
name  dir                                  2013-03-25 18:57 rwxr-xr-x   hive   supergroup

I want the table top be created as Owner = sasubramanian
How can I do that ?

Thanks
sanjay

CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.



--
Nitin Pawar



--
Nitin Pawar

CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.



--
Nitin Pawar

CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.



--
Nitin Pawar



--
Nitin Pawar

CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.

Mime
View raw message