hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <>
Subject [jira] [Created] (HIVE-10978) Document fs.trash.interval wrt Hive and HDFS Encryption
Date Wed, 10 Jun 2015 23:13:00 GMT
Eugene Koifman created HIVE-10978:

             Summary: Document fs.trash.interval wrt Hive and HDFS Encryption
                 Key: HIVE-10978
             Project: Hive
          Issue Type: Bug
          Components: Documentation, Security
    Affects Versions: 1.2.0
            Reporter: Eugene Koifman
            Priority: Critical

When HDFS is encrypted (TDE is enabled), DROP TABLE and DROP PARTITION have unexpected behavior
when Hadoop Trash feature is enabled.
The later is enabled by setting fs.trash.interval > 0 in core-site.xml.
When Trash is enabled, the data file for the table, should be "moved" to Trash bin. If the
table is inside an Encryption Zone, this "move" operation is not allowed.
There are 2 ways to deal with this:
1. use PURGE, as in DROP TABLE blah PURGE. This skips the Trash bin even if enabled.
2. set fs.trash.interval = 0. It is critical that this config change is done in core-site.xml.
Setting it in hive-site.xml may lead to very strange behavior where the table metadata is
deleted but the data file remains.  This will lead to data corruption if a table with the
same name is later created.

This message was sent by Atlassian JIRA

View raw message