Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 289FA10031 for ; Sat, 5 Oct 2013 09:50:28 +0000 (UTC) Received: (qmail 58686 invoked by uid 500); 5 Oct 2013 09:50:23 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 58615 invoked by uid 500); 5 Oct 2013 09:50:22 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 58606 invoked by uid 99); 5 Oct 2013 09:50:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Oct 2013 09:50:21 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of olorinbant@gmail.com designates 74.125.82.172 as permitted sender) Received: from [74.125.82.172] (HELO mail-we0-f172.google.com) (74.125.82.172) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Oct 2013 09:50:13 +0000 Received: by mail-we0-f172.google.com with SMTP id q58so235880wes.3 for ; Sat, 05 Oct 2013 02:49:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=b8wTsmc8YdeNhmwyHYITsdILszzGh5rzoxbKmbFieX4=; b=beybRkFAPJWUrOQCkXbbWSn+4dNuJOzTHB+GHjN0mX54VH+kpaIdY9R4J21rlhn8HV 7rEg8sesEeLByAgv0n9fYnPzeJz9sO0apc3l/yzKoXa2hWMc3l4QI3SUrjJHLPFE/PFx kxor4uDZEgTErEj9Hzt7c3oZ0WyVUR0/e+9Sz8dJ1rQ00/iCVK4KYczNM9QCmmTwZkHf 4vlsWIJKT0iUpJSvlhuzo1RLC5QWaDEMwQYWAWGQS/1BdGyp5wTqdF4YEP561YBa+wy2 eII2qZXzOrcB+exjpclWAKrx99zFHJc3o5f8cSe0T9o9pvRLiJToRxDclFtZMWR9XmOl k02w== X-Received: by 10.180.183.108 with SMTP id el12mr10798869wic.55.1380966593414; Sat, 05 Oct 2013 02:49:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.204.137 with HTTP; Sat, 5 Oct 2013 02:49:23 -0700 (PDT) In-Reply-To: References: From: Mikhail Antonov Date: Sat, 5 Oct 2013 02:49:23 -0700 Message-ID: Subject: Re: Need help with Installation hive 0.11 To: "user@hive.apache.org" Content-Type: multipart/alternative; boundary=001a11c353ac3f4bec04e7fb5770 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c353ac3f4bec04e7fb5770 Content-Type: text/plain; charset=ISO-8859-1 There's no need to disable Hive security, what you probably may want to do is do use default (database/metastore permission-based) type of authentication as suggested above (I myself use default/my custom authentication, but not HCatalog one). DefaultHiveAuthorizationProvider - that's the auth provider you may want to try, should work. 2013/10/5 Lefty Leverenz > I'm out of my depth, but based on existing documentation I'll make some > suggestions you could try while waiting for replies from people who > actually know about installation. > > Following Mikhail's lead, since this is an HCatalog security problem > you should start by seeing if HCatalog is installed. Then you could try > changing the value of hive.security.authorization.manager -- I found it > documented in two places: > > - Hive Configuration Properties > - HCatalog Storage Based Authorization > > The HCatalog authorization doc says you need to add the HCatalog jar to > the Hive classpath (add "export HIVE_AUX_JARS_PATH=<*path to hcatalog jar* > >" to hive-env.sh), but I'm not sure if that's needed now that HCatalog > has been integrated with Hive. > > Try changing hive.security.authorization.manager to its default value, > either the one listed in the Hive doc or the one in the HCatalog doc: > > - > org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider > - org.apache.hcatalog.security.HdfsAuthorizationProvider > > Or you could turn off hive.security.authorization.enabled by setting it to > false (see the HCatalog authorization doc). There's more information about > this property in the Hive Authorization doc's Prerequisitessection. > > Changing configuration property values is documented in Getting Started (Configuration > management overview) > as well as Configuring Hive > . > > Agreed, the documentation is confusing. We're trying to improve it. > > -- Lefty > > > On Sat, Oct 5, 2013 at 12:53 AM, Baahu wrote: > >> Lefty, >> Can the install instructions given at >> https://cwiki.apache.org/confluence/display/Hive/GettingStarted be >> followed as is? >> I see instructions to build from source code just after "Installing Hive >> from a Stable Release" section, which is kinda confusing.I bump in to the >> error mentioned above while I execute the instructions in "Running Hive" >> section. >> >> Thanks, >> Baahu >> >> >> >> >> On Sat, Oct 5, 2013 at 1:17 PM, Lefty Leverenz wrote: >> >>> Do you have any suggestions for improving the Hive wiki instructions? >>> >>> -- Lefty >>> >>> >>> On Fri, Oct 4, 2013 at 7:30 PM, Baahu wrote: >>> >>>> Apache hive. >>>> On Oct 5, 2013 1:06 AM, "Mikhail Antonov" wrote: >>>> >>>>> Are you using apache hive or one from cloudera/Horton works? >>>>> >>>>> On Oct 4, 2013, at 11:52, Baahu wrote: >>>>> >>>>> Hi Mikhail, >>>>> I followed the steps mentioned in the Apache hive website, found the >>>>> instructions not so helpful... Is there any good link which provides proper >>>>> installation steps. >>>>> >>>>> Thanks >>>>> On Oct 5, 2013 12:07 AM, "Mikhail Antonov" >>>>> wrote: >>>>> >>>>>> Looks like you have StorageDelegationAuthorizationProvider enabled >>>>>> in hive-site.xml (like below), but hcatalog isn't installed somehow? >>>>>> >>>>>> >>>>>> hive.security.authorization.manager >>>>>> org.apache.hcatalog.security.StorageDelegationAuthorizationProvider >>>>>> the hive client authorization manager class name. >>>>>> The user defined authorization class should implement interface org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider. >>>>>> HCatalog uses a model, where authorization checks are delegated to the storage layer (hdfs, hbase, ...). >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 2013/10/4 Baahu >>>>>> >>>>>>> Hi, >>>>>>> Have installed Hive 0.11 in centos.I see the below error when I >>>>>>> execute hive.Can you please help me resolving this. >>>>>>> >>>>>>> [root@centosvm bin]# hive >>>>>>> 13/10/04 09:05:35 WARN conf.HiveConf: DEPRECATED: Configuration >>>>>>> property hive.metastore.local no longer has any effect. Make sure to >>>>>>> provide a valid value for hive.metastore.uris if you are connecting to a >>>>>>> remote metastore. >>>>>>> >>>>>>> Logging initialized using configuration in >>>>>>> jar:file:/opt/hive-0.11.0-bin/lib/hive-common-0.11.0.jar!/hive-log4j.properties >>>>>>> Hive history >>>>>>> file=/tmp/root/hive_job_log_root_8553@centosvm_201310040905_1657489660.txt >>>>>>> Exception in thread "main" java.lang.RuntimeException: >>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException: >>>>>>> java.lang.ClassNotFoundException: >>>>>>> org.apache.hcatalog.security.StorageDelegationAuthorizationProvider >>>>>>> at >>>>>>> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:278) >>>>>>> at >>>>>>> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670) >>>>>>> at >>>>>>> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) >>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>> Method) >>>>>>> at >>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>>>> at >>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:160) >>>>>>> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: >>>>>>> java.lang.ClassNotFoundException: >>>>>>> org.apache.hcatalog.security.StorageDelegationAuthorizationProvider >>>>>>> at >>>>>>> org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:342) >>>>>>> at >>>>>>> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:272) >>>>>>> ... 7 more >>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>> org.apache.hcatalog.security.StorageDelegationAuthorizationProvider >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> at java.lang.Class.forName0(Native Method) >>>>>>> at java.lang.Class.forName(Class.java:270) >>>>>>> at >>>>>>> org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:335) >>>>>>> ... 8 more >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> Michael Antonov >>>>>> >>>>> >>> >> >> >> -- >> Twitter:http://twitter.com/Baahu >> >> > -- Thanks, Michael Antonov --001a11c353ac3f4bec04e7fb5770 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
There's no need to disable Hive security, what you pro= bably may want to do is do use default (database/metastore permission-based= ) type of authentication as suggested above (I myself use default/my custom= authentication, but not HCatalog one).=A0DefaultHiveAuthorizationProvider - that's the auth p= rovider you may want to try, should work.






2013/10/5 Lefty Levere= nz <leftyleverenz@gmail.com>
I'm out of my dept= h, but based on existing documentation I'll make some suggestions you c= ould try while waiting for replies from people who actually know about inst= allation.

Following Mikhail's lead, since this is an HCatalog security probl= em you=A0should start by seeing if HCatalog is installed. =A0Then you=A0cou= ld try changing the value of hive.security.authorization.manager -- I found= it documented in two places:
The HCatalog authorization doc says you need to add the HCa= talog jar to the Hive classpath (add "export HIVE_AUX_JARS_PATH=3D<path to hcatalog jar&g= t;" to hive-env.sh), but I'm not sure if that's needed = now that HCatalog has been integrated with Hive.=A0

Try changing hive.security.authorization.manager to its= default value, either the one listed in the Hive doc=A0or the one in the H= Catalog doc:=A0
  • org.apache.hadoop.hive.ql.security.author= ization.DefaultHiveAuthorizationProvider
  • org.apache.hcatalog.security.HdfsAuthorizationProvider=A0
Or you could turn off hive.security.authorization.enabled by setting= it to false (see the HCatalog authorization doc). =A0There's more info= rmation about this property in the Hive Authorization doc's Prerequisites section.

Changing configuration property values is documen= ted in Getting Started (Configuration management overview) as well as Configuring Hi= ve.

Agreed, the documentation is confusing. =A0We're tr= ying to improve it.

-- Lefty


On Sat, Oct 5, 2013 at 12:53 AM, Baahu <= span dir=3D"ltr"><bahubali@gmail.com> wrote:
Lefty,
Can the install instruction= s given at https://cwiki.apache.org/confluence/display/H= ive/GettingStarted be followed as is?
I see instructions to build from source code just after "Installing Hi= ve from a Stable Release" section, which is kinda confusing.I bump in = to the error mentioned above while I execute the instructions in "Runn= ing Hive" section.

Thanks,
Baahu




On Sat= , Oct 5, 2013 at 1:17 PM, Lefty Leverenz <leftyleverenz@gmail.com> wrote:
Do you have any suggestions for improving the Hive wiki instructions?
=

-- Lefty


On Fri, Oct 4, 2013 at 7:30 PM, Baahu <bahubali@gmail.com> wrote:

Apache hive.

On Oct 5, 2013 1:06 AM, "Mikhail Antonov&qu= ot; <olorinban= t@gmail.com> wrote:
Are you using apache hive or one from cloudera/Horto= n works?

On Oct 4, 2013, at 11:52, Baahu <bahubali@gmail.com> wrote:

Hi Mikhail,
I followed the steps mentioned in the Apache hive website, found the instru= ctions not so helpful... Is there any good link which provides proper insta= llation steps.

Thanks

On Oct 5, 2013 12:07 AM, "Mikhail Antonov&q= uot; <olorinba= nt@gmail.com> wrote:
Looks like you have=A0StorageDelegationAuthorizationProvider enabled in hive-site= .xml (like below), but hcatalog isn't installed somehow?
<property>
  <name>hive.security.authorization.manager</name>
  <value>org.apache.hcatalog.security.StorageDelegationAuthorizationP=
rovider</value>
  <description>the hive client authorization manager class name.
  The user defined authorization class should implement interface org.apach=
e.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
  HCatalog uses a model, where authorization checks are delegated to the st=
orage layer (hdfs, hbase, ...).
  </description>
</property>


2013/10/4 Baahu <bahubali@gmail.com><= br>
Hi,
Have installed Hive 0.11 in centos.I see= the below error when I execute hive.Can you please help me resolving this.=

[root@centosvm bin]# hive
13/10/04 = 09:05:35 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metast= ore.local no longer has any effect. Make sure to provide a valid value for = hive.metastore.uris if you are connecting to a remote metastore.

Logging initialized using configuration in jar:file:/opt/hive-0.11.0-bi= n/lib/hive-common-0.11.0.jar!/hive-log4j.properties
Hive history file=3D= /tmp/root/hive_job_log_root_8553@centosvm_201310040905_1657489660.txt
Exception in thread "main" java.lang.RuntimeException: org.apache= .hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: o= rg.apache.hcatalog.security.StorageDelegationAuthorizationProvider
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.hive.ql.session.SessionState.sta= rt(SessionState.java:278)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver= .java:670)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.hive.cli.CliDriver= .main(CliDriver.java:614)
=A0=A0=A0=A0=A0=A0=A0 at sun.reflect.NativeMet= hodAccessorImpl.invoke0(Native Method)
=A0=A0=A0=A0=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke(Native= MethodAccessorImpl.java:57)
=A0=A0=A0=A0=A0=A0=A0 at sun.reflect.Delegat= ingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
=A0= =A0=A0=A0=A0=A0=A0 at java.lang.reflect.Method.invoke(Method.java:606)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:160= )
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang= .ClassNotFoundException: org.apache.hcatalog.security.StorageDelegationAuth= orizationProvider
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAu= thorizeProviderManager(HiveUtils.java:342)
=A0=A0=A0=A0=A0=A0=A0 at org.= apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:272)
= =A0=A0=A0=A0=A0=A0=A0 ... 7 more
Caused by: java.lang.ClassNotFoundExcep= tion: org.apache.hcatalog.security.StorageDelegationAuthorizationProvider =A0=A0=A0=A0=A0=A0=A0 at java.net.URLClassLoader$1.run(URLClassLoader.java:= 366)
=A0=A0=A0=A0=A0=A0=A0 at java.net.URLClassLoader$1.run(URLClassLoad= er.java:355)
=A0=A0=A0=A0=A0=A0=A0 at java.security.AccessController.doP= rivileged(Native Method)
=A0=A0=A0=A0=A0=A0=A0 at java.net.URLClassLoade= r.findClass(URLClassLoader.java:354)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.ClassLoader.loadClass(ClassLoader.java:4= 24)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.ClassLoader.loadClass(ClassLoader= .java:357)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.Class.forName0(Native Meth= od)
=A0=A0=A0=A0=A0=A0=A0 at java.lang.Class.forName(Class.java:270)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAu= thorizeProviderManager(HiveUtils.java:335)
=A0=A0=A0=A0=A0=A0=A0 ... 8 m= ore

Thanks.



--
Thanks,
M= ichael Antonov




--
Twitter:http://twitter.com/Baahu




--
Thanks,
M= ichael Antonov --001a11c353ac3f4bec04e7fb5770--