hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdelrahman Shettia <ashet...@hortonworks.com>
Subject Re: Permission denied creating external table
Date Mon, 24 Mar 2014 16:21:59 GMT
Hi Rishabh, 

Not sure if this entirly right. I did a quick test and I can create an external table without
the write permission. 

hive> CREATE EXTERNAL TABLE mytest (name string) location '/test/mytest';
OK
Time taken: 1.675 seconds
hive> dfs -ls /test ; 
Found 1 items
drwxr-xr-x   - hive hdfs          0 2014-03-24 09:03 /test/mytest
hive> exit;    
[hive@sandbox ~]$ hadoop dfs -ls /test
Found 1 items
drwxr-xr-x   - hive hdfs          0 2014-03-24 09:03 /test/mytest
[hive@sandbox ~]$ hadoop dfs -ls / | grep test
drwxr-xr-x   - hdfs   hdfs            0 2014-03-24 09:03 /test

So, It depends on the configurations in my previous email and the proxy settings for Hive
users in core-site.xml.: 
<property>
  <name>hadoop.proxyuser.hive.groups</name>
  <value>users</value>
  <description>Allow the superuser hive to impersonate any members of the group users.
Required only when installing Hive.
  </description>
</property>
where $HIVE_USER is the user owning Hive Services. For example, hive.

<property>
  <name>hadoop.proxyuser.hive.hosts</name>
  <value>$Hive_Hostname_FQDN</value>
  <description>Hostname from where superuser hive can connect. Required only when installing
Hive.
  </description>
</property>

Thanks,
Rahman

On Mar 24, 2014, at 8:55 AM, Rishabh Bhardwaj <rbnext29@yahoo.com> wrote:

> Hi Oliver,
> In order to create external tables you must be having write access to the folder.
> Also in order to create external table just give the location of the folder in which
your file is located.
> 
> For example if your file "actual_log_file_name.seq" is stored in "03-24" folder then
try one attempt as:
> CREATE EXTERNAL TABLE mylogs (line STRING) STORED AS SEQUENCEFILE LOCATION 'hdfs:///logs/2014/03-24/
> 
> 
> Otherwise I too has the same problem and I found out that without write permission user
can not create external table.Other alternate is to copy this file under your hdfs directory
then create a table (not every time useful).
> 
> Thanks,
> Rishabh.
> 
> 
> On Monday, 24 March 2014 9:16 PM, Abdelrahman Shettia <ashettia@hortonworks.com>
wrote:
> Hi Oliver, 
> 
> Can you perform a simple test of hadoop fs -cat hdfs:///logs/2014/03-24/actual_log_file_name.seq
by the same user? Also what are the configurations setting for the following?
> 
> hive.metastore.execute.setugi
> 
> hive.metastore.warehouse.dir
> 
> hive.metastore.uris 
> 
> Thanks,
> Rahman
> 
> On Mar 24, 2014, at 8:17 AM, Oliver <ohookins@gmail.com> wrote:
> 
>> Hi,
>> 
>> I have a bunch of data already in place in a directory on HDFS containing many different
logs of different types, so I'm attempting to load these externally like so:
>> 
>> CREATE EXTERNAL TABLE mylogs (line STRING) STORED AS SEQUENCEFILE LOCATION 'hdfs:///logs/2014/03-24/actual_log_file_name.seq';
>> 
>> However I get this error back when doing so:
>> 
>> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission
denied: user=oliver, access=WRITE, inode="/logs/2014/03-24":logs:supergroup:drwxr-xr-x
>> 	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:204)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4716)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4698)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4672)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3035)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:2999)
>> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2980)
>> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:648)
>> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:419)
>> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44970)
>> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
>> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
>> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
>> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
>> 	at java.security.AccessController.doPrivileged(Native Method)
>> 	at javax.security.auth.Subject.doAs(Unknown Source)
>> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
>> )
>> 
>> This directory is intentionally read-only by regular users who want to read the logs
and analyse them. Am I missing some configuration data for Hive that will tell it to only
store metadata elsewhere? I already have hive.metastore.warehouse.dir set to another location
where I have write permission.
>> 
>> Best Regards,
>> Oliver
> 
> 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it
is addressed and may contain information that is confidential, privileged and exempt from
disclosure under applicable law. If the reader of this message is not the intended recipient,
you are hereby notified that any printing, copying, dissemination, distribution, disclosure
or forwarding of this communication is strictly prohibited. If you have received this communication
in error, please contact the sender immediately and delete it from your system. Thank You.
> 
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message