hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tale Firefly <tale.h...@gmail.com>
Subject HiveServer2 - Serious problem - Permission denied with another user than the one I am using
Date Tue, 03 May 2016 07:53:51 GMT
Hello !

I send you this mail because I have a problem with a table in Hive through
HiveServer2 (HTTP or thrift).

I'm using an HDP 2.3.2 with Hive 1.2.1.

The problem is the following.

I have the following table :
###
CREATE TABLE `mytable`(
    `field1` timestamp,
    `field2` string,
    `field3` bigint,
    `field4` varchar(40),
    `field5` int,
    `field6` int,
    `field7` char(17),
    `field8` varchar(40),
    `field9` varchar(15),
    `field10` varchar(100),
    `field11` varchar(30),
    `field12` varchar(64),
    `field13` varchar(64),
    `field14` varchar(15),
    `field15` varchar(25),
    `field16` varchar(3),
    `field17` varchar(512))
PARTITIONED BY (
    `year` string,
    `month` string,
    `day` string)
ROW FORMAT SERDE
    'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
    'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
    'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION
    'hdfs://<NAMENODE>/apps/hive/warehouse/mydb.db/mytable'
TBLPROPERTIES (
    'orc.compress'='SNAPPY',
    'transient_lastDdlTime'='1461067777')
###

Note : I'm using Storage permissions for Hive.

Then I performed the following query on this table with my user user1:
###
SELECT * FROM `mydb.mytable` WHERE `year` = '2016' AND `month` = '04' AND
`day` = '27' LIMIT 250
###

And I got the following error message :
###
java.io.IOException: java.lang.RuntimeException: serious problem
###

When I dig more in the hiveserver2.log, I can find the following error
message :
###
16-04-28 11:01:53,649 WARN [HiveServer2-Handler-Pool: Thread-46]:
thrift.ThriftCLIService (ThriftCLIService.java:FetchResults(681)) - Error
fetching results:
org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
java.lang.RuntimeException: serious problem
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:352)
(...)
Caused by: java.io.IOException: java.lang.RuntimeException: serious problem
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
(...)
... 24 more
Caused by: java.lang.RuntimeException: serious problem
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1025)
(...)
... 28 more
Caused by: java.util.concurrent.ExecutionException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=user2, access=EXECUTE,
inode="/apps/hive/warehouse/mydb.db/mytable/year=2016/month=04/day=27":user1:group1:drwxr-x---
###

user2 is not a member of group1, so it is normal that this error occures.
But I was doing my query with my user user1, not user2.

I checked then the hdfs audit logs about my user user1 and about the other
user user2.
I didn't find any activity from the user user2 for this day, except at the
exact same time.
###
2016-04-28 11:01:53,649 INFO FSNamesystem.audit: allowed=false ugi=user2
(auth:PROXY) via hive/<hiveserver2-hostname>@<kerberos-realm>
(auth:KERBEROS) ip=/<IP> cmd=listStatus
src=/apps/hive/warehouse/mydb.db/mytable/year=2016/month=04/day=27 dst=null
perm=null proto=rpc
###

This problem does not occure when I use hiveCLI or when I change the
storage format of the table from ORC to another one.

I was wondering if anyone already encountered this problem before ? This is
really a blocking point here and I was wondering how to solve it ?

I found the following JIRA for Hive :
https://issues.apache.org/jira/browse/HIVE-13120

But it didn't say which version is affected, it only says when it is fixed.
And I'm not completely sure this is my problem ?

So my idea was to try to modify hive.fetch.task.conversion and set it to
"none", in order to try to solve this issue, but it will have as
consequence that listing even the smallest table will launch a job in Yarn,
which is not cool at all.

I'm open to all advices concerning this problem.
What is the best practice :
- Use ORC format and deactivate hive.fetch.task.conversion ?
- Or use another format and keep hive.fetch.task.conversion set to its
default value ?
- Or another solution ?

Hope you can help me.

BR.

Tale

Mime
View raw message