hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10978) Document fs.trash.interval wrt Hive and HDFS Encryption
Date Fri, 28 Aug 2015 17:01:46 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720235#comment-14720235
] 

Eugene Koifman commented on HIVE-10978:
---------------------------------------

I think mentioning this in Drop Table/Partition section is a good idea, but the most critical
part is that " fs.trash.interval" has to be set in core-site.xml (i.e. the hadoop config file)
not any hive-site.xml or at CLI.  

> Document fs.trash.interval wrt Hive and HDFS Encryption
> -------------------------------------------------------
>
>                 Key: HIVE-10978
>                 URL: https://issues.apache.org/jira/browse/HIVE-10978
>             Project: Hive
>          Issue Type: Bug
>          Components: Documentation, Security
>    Affects Versions: 1.2.0
>            Reporter: Eugene Koifman
>            Priority: Critical
>              Labels: TODOC1.2
>
> This should be documented in 1.2.1 Release Notes
> When HDFS is encrypted (TDE is enabled), DROP TABLE and DROP PARTITION have unexpected
behavior when Hadoop Trash feature is enabled.
> The later is enabled by setting fs.trash.interval > 0 in core-site.xml.
> When Trash is enabled, the data file for the table, should be "moved" to Trash bin. If
the table is inside an Encryption Zone, this "move" operation is not allowed.
> There are 2 ways to deal with this:
> 1. use PURGE, as in DROP TABLE blah PURGE. This skips the Trash bin even if enabled.
> 2. set fs.trash.interval = 0. It is critical that this config change is done in core-site.xml.
Setting it in hive-site.xml may lead to very strange behavior where the table metadata is
deleted but the data file remains.  This will lead to data corruption if a table with the
same name is later created.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message