hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-23968) CTAS with TBLPROPERTIES ('transactional'='false') does not entertain translated table location
Date Thu, 19 Nov 2020 00:42:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23968?focusedWorklogId=513887&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-513887
]

ASF GitHub Bot logged work on HIVE-23968:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 19/Nov/20 00:41
            Start Date: 19/Nov/20 00:41
    Worklog Time Spent: 10m 
      Work Description: github-actions[bot] closed pull request #1348:
URL: https://github.com/apache/hive/pull/1348


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 513887)
    Time Spent: 20m  (was: 10m)

> CTAS with TBLPROPERTIES ('transactional'='false') does not entertain translated table
location
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-23968
>                 URL: https://issues.apache.org/jira/browse/HIVE-23968
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 4.0.0
>            Reporter: Rajkumar Singh
>            Assignee: Rajkumar Singh
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> HMS translation layer convert the table to external based on the transactional property
set to false but MoveTask does not entertain the translated table location and move the data
to the managed table location;
> steps to repro:
> {code:java}
> create table nontxnal TBLPROPERTIES ('transactional'='false') as select * from abc;
> {code}
> select query on table return nothing t but the source table has data in it.
> {code:java}
> select * from nontxnal;
> +--------------+
> | nontxnal.id  |
> +--------------+
> +--------------+
> {code}
> --show create table
> {code:java}
> CREATE EXTERNAL TABLE `nontxnal`(                  |
> |   `id` int)                                        |
> | ROW FORMAT SERDE                                   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'      |
> | STORED AS INPUTFORMAT                              |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  |
> | OUTPUTFORMAT                                       |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION                                           |
> |   'hdfs://hostname:8020/warehouse/tablespace/external/hive/nontxnal' |
> | TBLPROPERTIES (                                    |
> |   'TRANSLATED_TO_EXTERNAL'='TRUE',                 |
> |   'bucketing_version'='2',                         |
> |   'external.table.purge'='TRUE',                   |
> |   'transient_lastDdlTime'='1596215634')            |
> {code}
> table data is moved to the managed location:
> ```
> dfs -ls -R  hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> +----------------------------------------------------+
> |                     DFS Output                     |
> +----------------------------------------------------+
> | -rw-rw----+  3 hive hadoop        201 2020-07-31 17:05 hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal/000000_0
|
> +----------------------------------------------------+
> ```
> The problem seems to be here
> isExternal evaluates to false since the statement is missing external
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L446
> and location return to the managed location
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L455



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message