hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexandre Linte (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-15128) "hive.exec.stagingdir" produces an "AccessControlException" when working with temp tables
Date Fri, 04 Nov 2016 10:28:59 GMT
Alexandre Linte created HIVE-15128:
--------------------------------------

             Summary: "hive.exec.stagingdir" produces an "AccessControlException" when working
with temp tables
                 Key: HIVE-15128
                 URL: https://issues.apache.org/jira/browse/HIVE-15128
             Project: Hive
          Issue Type: Bug
          Components: Beeline, CLI, Hive
    Affects Versions: 2.1.0
         Environment: The property "hive.exec.stagingdir" is producing an AccessControlException
error, only when working with a temporary table and with an INSERT INTO TABLE statement.
The configuration used in the hive-site.xml is:
{noformat}
  <property>
      <name>hive.exec.stagingdir</name>
      <value>/user/${user.name}/.hiveStaging/.staging</value>
      <description>In Hive >= 0.14, set to ${hive.exec.scratchdir}/${user.name}/.staging</description>
  </property>
{noformat}

Trying to insert a value into a temporary table will produce the following output:
{noformat}
hive (shfs3453)> INSERT INTO TABLE test_table_tmp VALUES (11101, 'John', 'Oakland');
Query ID = shfs3453_20161103180522_5fceae9b-6fb5-4cdd-a0d9-37cc93b27a9f
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1478096256452_0018)

----------------------------------------------------------------------------------------------
        VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container     SUCCEEDED      1          1        0        0       0     
 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 16.09 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
Failed with exception org.apache.hadoop.security.AccessControlException: User does not belong
to hadoop
        at org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. org.apache.hadoop.security.AccessControlException:
User does not belong to hadoop
        at org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:86)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1676)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:702)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:464)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
{noformat}

But if you do an INSERT OVERWRITE TABLE, there will be a warning only. No AccessControlException
error here and erverything is properly inserted.

{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John', 'Oakland');
Query ID = shfs3453_20161103180603_d8d5f689-cc55-44d0-b160-bba0df8d186a
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1478096256452_0018)

----------------------------------------------------------------------------------------------
        VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container     SUCCEEDED      2          2        0        0       0     
 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 6.74 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
chgrp: changing ownership of 'hdfs://sandbox/tmp/hive/shfs3453/edab56b7-b1e1-4f87-bdfd-eeece41f6fc9/_tmp_space.db/874bfdbd-7bab-4e0c-85ac-ec4552ea560b':
User does not belong to hadoop
OK
Time taken: 8.224 seconds
{noformat}

If the previous query is done one more time, the warning will not be displayed.

{noformat}
hive (shfs3453)> INSERT OVERWRITE TABLE test_table_tmp VALUES (11101, 'John', 'Oakland');
Query ID = shfs3453_20161103180831_40350dc3-1f66-45e2-8304-90b00933aead
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1478096256452_0018)

----------------------------------------------------------------------------------------------
        VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container     SUCCEEDED      1          1        0        0       0     
 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 13.00 s
----------------------------------------------------------------------------------------------
Loading data to table shfs3453.test_table_tmp
OK
Time taken: 14.161 seconds
{noformat}

Note: if the warning is displayed one time, an INSERT INTO TABLE will work.
            Reporter: Alexandre Linte






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message