Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9226DF3C4 for ; Tue, 26 Mar 2013 02:55:17 +0000 (UTC) Received: (qmail 40361 invoked by uid 500); 26 Mar 2013 02:55:15 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 40298 invoked by uid 500); 26 Mar 2013 02:55:15 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 40286 invoked by uid 99); 26 Mar 2013 02:55:15 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Mar 2013 02:55:15 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of Sanjay.Subramanian@wizecommerce.com designates 213.199.154.186 as permitted sender) Received: from [213.199.154.186] (HELO db8outboundpool.messaging.microsoft.com) (213.199.154.186) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Mar 2013 02:55:08 +0000 Received: from mail118-db8-R.bigfish.com (10.174.8.245) by DB8EHSOBE025.bigfish.com (10.174.4.88) with Microsoft SMTP Server id 14.1.225.23; Tue, 26 Mar 2013 02:54:46 +0000 Received: from mail118-db8 (localhost [127.0.0.1]) by mail118-db8-R.bigfish.com (Postfix) with ESMTP id 94AC4401AD for ; Tue, 26 Mar 2013 02:54:46 +0000 (UTC) X-Forefront-Antispam-Report: CIP:157.56.232.197;KIP:(null);UIP:(null);IPV:NLI;H:BLUPRD0411HT001.namprd04.prod.outlook.com;RD:none;EFVD:NLI X-SpamScore: -5 X-BigFish: PS-5(zz98dI9371I936eI181fMc85eh9a6kzz1f42h1ee6h1de0h1202h1e76h1d1ah1d2ahzz8275dh18c673h8275bhz2fh2a8h668h839hbe3he5bhf0ah1288h12a5h12bdh137ah1441h1504h1537h153bh162dh1631h1758h18e1h1946h19b5h1ad9h1b0ah1bceh1155h) Received-SPF: pass (mail118-db8: domain of wizecommerce.com designates 157.56.232.197 as permitted sender) client-ip=157.56.232.197; envelope-from=Sanjay.Subramanian@wizecommerce.com; helo=BLUPRD0411HT001.namprd04.prod.outlook.com ;.outlook.com ; Received: from mail118-db8 (localhost.localdomain [127.0.0.1]) by mail118-db8 (MessageSwitch) id 136426648472606_23799; Tue, 26 Mar 2013 02:54:44 +0000 (UTC) Received: from DB8EHSMHS016.bigfish.com (unknown [10.174.8.225]) by mail118-db8.bigfish.com (Postfix) with ESMTP id 0E1BA3E004D for ; Tue, 26 Mar 2013 02:54:44 +0000 (UTC) Received: from BLUPRD0411HT001.namprd04.prod.outlook.com (157.56.232.197) by DB8EHSMHS016.bigfish.com (10.174.4.26) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 26 Mar 2013 02:54:43 +0000 Received: from BLUPRD0411MB426.namprd04.prod.outlook.com ([169.254.10.122]) by BLUPRD0411HT001.namprd04.prod.outlook.com ([10.255.127.36]) with mapi id 14.16.0275.006; Tue, 26 Mar 2013 02:54:37 +0000 From: Sanjay Subramanian To: "user@hive.apache.org" Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner ? Thread-Topic: HDFS directory in /user/hive/warehouse getting "hive" as Owner ? Thread-Index: AQHOKcdDQFBc7E4wqUqDimJkb1yySZi3O1qA//+MLQCAAHxggP//jbCA Date: Tue, 26 Mar 2013 02:54:37 +0000 Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.255.127.4] Content-Type: multipart/alternative; boundary="_000_CD765AEE5F43sanjaysubramanianwizecommercecom_" MIME-Version: 1.0 X-OriginatorOrg: wizecommerce.com X-Virus-Checked: Checked by ClamAV on apache.org --_000_CD765AEE5F43sanjaysubramanianwizecommercecom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable I am using Hive Version: 0.9.0+155-1.cdh4.1.2.p0.21~precise-cdh4.1.2 My metastore is MySQL My hive.security.authorization.enabled is set to false as of now=85I am not= able to add partitions if I keep that as true hive.security.authorization.enabled false enable or disable the hive client authorization I have not defined a property=3Dhive.security.authorization.manager Thanks sanjay From: Nitin Pawar > Reply-To: "user@hive.apache.org" > Date: Monday, March 25, 2013 7:43 PM To: "user@hive.apache.org" > Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner= ? YARN should not play any role in any create table statement. It just create= s a directory with DFSClient. Normally it tries to create it with the cli u= serid in my experience. which version of hive are you using? which is your metastore? Can you check for the following values? hive.security.authorization.enabled true enable or disable the hive client authorization hive.security.authorization.manager org.apache.hcatalog.security.HdfsAuthorizationProvider the hive client authorization manager class name. The user defined authorization class should implement interface org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvi= der. On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subramanian > wrote: Hi Nitin I notice this peculiarity in Yarn and Hive I have another earlier cluster with MRv1 where I have created and run sever= al hive tables and scripts ; The same test Create Table script gives the co= rrect owner name I added location but that did not help hive -e "CREATE TABLE name (id INT, name STRING) LOCATION '/user/hive/ware= house/name';" Thanks Sanjay From: Nitin Pawar > Reply-To: "user@hive.apache.org" > Date: Monday, March 25, 2013 7:13 PM To: "user@hive.apache.org" > Subject: Re: HDFS directory in /user/hive/warehouse getting "hive" as Owner= ? Forgot to add, if you want full filesystem level security on HDFS then you will need to en= able kerberos based security. On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar > wrote: Sanjay, can you try adding 'LOCATION' clause to your create statement. By default the hive warehouse directory is writable by all the user. To cre= ate it by the individual users you need to provide by the location clause. On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subramanian > wrote: Steps to recreate the use case: - Log in as sasubramanian to Linux Box - Execute hive -e "CREATE TABLE name (id INT, name STRING);" - Go to HDFS /user/hive/warehouse/ Name Type Size Replication Block Size Modification Time Permissio= n Owner Group name dir 2013-03-25 18:57 rwxr-xr-x hiv= e supergroup I want the table top be created as Owner =3D sasubramanian How can I do that ? Thanks sanjay CONFIDENTIALITY NOTICE =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This email message and any attachments are for the exclusive use of the int= ended recipient(s) and may contain confidential and privileged information.= Any unauthorized review, use, disclosure or distribution is prohibited. If= you are not the intended recipient, please contact the sender by reply ema= il and destroy all copies of the original message along with any attachment= s, from your computer system. If you are the intended recipient, please be = advised that the content of this message is subject to access, review and d= isclosure by the sender's Email System Administrator. -- Nitin Pawar -- Nitin Pawar CONFIDENTIALITY NOTICE =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This email message and any attachments are for the exclusive use of the int= ended recipient(s) and may contain confidential and privileged information.= Any unauthorized review, use, disclosure or distribution is prohibited. If= you are not the intended recipient, please contact the sender by reply ema= il and destroy all copies of the original message along with any attachment= s, from your computer system. If you are the intended recipient, please be = advised that the content of this message is subject to access, review and d= isclosure by the sender's Email System Administrator. -- Nitin Pawar CONFIDENTIALITY NOTICE =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This email message and any attachments are for the exclusive use of the int= ended recipient(s) and may contain confidential and privileged information.= Any unauthorized review, use, disclosure or distribution is prohibited. If= you are not the intended recipient, please contact the sender by reply ema= il and destroy all copies of the original message along with any attachment= s, from your computer system. If you are the intended recipient, please be = advised that the content of this message is subject to access, review and d= isclosure by the sender's Email System Administrator. --_000_CD765AEE5F43sanjaysubramanianwizecommercecom_ Content-Type: text/html; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable
I am using 
Hive Version: 0.9.0+155-1.cdh4.1.2.p0.2= 1~precise-cdh4.1.2
My metastore is MySQL 
My hive.security.authorization.enabled is set to false as o=
f now=85I am not able to add partitions if I keep that as true
 <property>
    <name>hive.security.authorization.enabled</name>
    <value>false</value>
    <description>enable or disable the hive client authorization</=
description>
  </property>
I have not defined a property=3Dhive= .security.authorization.manager

Thanks
sanjay

From: Nitin Pawar <nitinpawar432@gmail.com>
Reply-To: "user@hive.apache.org" <user@hive.apache.org>
Date: Monday, March 25, 2013 7:43 P= M
To: "user@hive.apache.org" <user@hive.apache.org>
Subject: Re: HDFS directory in /use= r/hive/warehouse getting "hive" as Owner ?

YARN should not play any role in any create table statemen= t. It just creates a directory with DFSClient. Normally it tries to create = it with the cli userid in my experience.

which version of hive are you using? 
which is your metastore? 

Can you check for the following values? 
 <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</=
description>
  </property>

  <property>
    <name>hive.security.authorization.manager</name>
    <value>org.apache.hcatalog.security.HdfsAuthorizationProvider<=
/value>
    <description>the hive client authorization manager class name.
    The user defined authorization class should implement =
interface=20
    org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvi=
der.
    </description>
  </property>


On Tue, Mar 26, 2013 at 7:48 AM, Sanjay Subraman= ian <Sanjay.Subramanian@wizecommerce.com> wrote:
Hi Nitin
I notice this peculiarity in Yarn and Hive
I have another earlier cluster with MRv1 where I have created and run = several hive tables and scripts ; The same test Create Table script gives t= he correct owner name
I added location but that did not help 
hive -e "CREATE TABLE name (id INT,  name STRING) LOCATION '= /user/hive/warehouse/name';"

Thanks
Sanjay

From: Nitin Pawar <nitinpawar432@gmail.com= >
Reply-To: "user@hive.apache.org" <<= a href=3D"mailto:user@hive.apache.org" target=3D"_blank">user@hive.apache.o= rg>
Date: Monday, March 25, 2013 7:13 P= M
To: "user@hive.apache.org" <user@hive.apache.org= >
Subject: Re: HDFS directory in /use= r/hive/warehouse getting "hive" as Owner ?

Forgot to add, 
if you want full filesystem level security on HDFS then you will need = to enable kerberos based security. 


On Tue, Mar 26, 2013 at 7:41 AM, Nitin Pawar <nitinpawar= 432@gmail.com> wrote:
Sanjay, 

can you try adding 'LOCATION' clause to your create statement. 
By default the hive warehouse directory is writable by all the user. T= o create it by the individual users you need to provide by the location cla= use. 


On Tue, Mar 26, 2013 at 7:31 AM, Sanjay Subraman= ian <Sanjay.Subramanian@wizecommerce.com> wrote:
Steps to recreate the use case:

- Log in as sasubramanian to Linux = Box 
- Execute hive -e "CREATE TABLE name = (id INT,  name STRING);"
- Go to HDFS /user/hive/warehouse/

Name  Type  Size  Replication &n= bsp;Block Size  Modification Time      Permission  = ;Owner  Group  
name  dir         &nbs= p;                     &n= bsp;  2013-03-25 18:57&nbs= p;rwxr-xr-x   hive   supergroup 

I want the ta= ble top be created as Owner =3D sasubramanian
How can I do = that ?

Thanks
sanjay

CONFIDENTIALITY NOTICE
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This email message and any attachments are for the exclusive use of the int= ended recipient(s) and may contain confidential and privileged information.= Any unauthorized review, use, disclosure or distribution is prohibited. If= you are not the intended recipient, please contact the sender by reply email and destroy all copies of the ori= ginal message along with any attachments, from your computer system. If you= are the intended recipient, please be advised that the content of this mes= sage is subject to access, review and disclosure by the sender's Email System Administrator.



--
Nitin Pawar



--
Nitin Pawar

CONFIDENTIALITY NOTICE
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This email message and any attachments are for the exclusive use of the int= ended recipient(s) and may contain confidential and privileged information.= Any unauthorized review, use, disclosure or distribution is prohibited. If= you are not the intended recipient, please contact the sender by reply email and destroy all copies of the ori= ginal message along with any attachments, from your computer system. If you= are the intended recipient, please be advised that the content of this mes= sage is subject to access, review and disclosure by the sender's Email System Administrator.



--
Nitin Pawar

CONFIDENTIALITY NOTICE
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This email message and any attachments are for the exclusive use of the int= ended recipient(s) and may contain confidential and privileged information.= Any unauthorized review, use, disclosure or distribution is prohibited. If= you are not the intended recipient, please contact the sender by reply email and destroy all copies of the ori= ginal message along with any attachments, from your computer system. If you= are the intended recipient, please be advised that the content of this mes= sage is subject to access, review and disclosure by the sender's Email System Administrator.
--_000_CD765AEE5F43sanjaysubramanianwizecommercecom_--