hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <rajesh.balamo...@gmail.com>
Subject Re: Review Request 50434: HIVE-14323: Reduce number of FS permissions and redundant FS operations
Date Mon, 01 Aug 2016 08:32:01 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50434/
-----------------------------------------------------------

(Updated Aug. 1, 2016, 8:32 a.m.)


Review request for hive, Ashutosh Chauhan and Chris Nauroth.


Bugs: HIVE-14323
    https://issues.apache.org/jira/browse/HIVE-14323


Repository: hive-git


Description
-------

HIVE-14323: Reduce number of FS permissions and redundant FS operations

1. Honor HIVE_WAREHOUSE_SUBDIR_INHERIT_PERMS when getting stage directory in Context and in
mkdir() in FileUtils
2. Reduce FS calls like exists()-->delete() to just delete().
3. Similar to #2 in Hive.replaceFiles()
4. Refactor Hive.trashFiles() so that it can make use of FileStatus information if available
(s3 FS lookups are expensive)
5. Refactor Hive.removeTempOrDuplicateFiles() so that it can make use of FileStatus information
if available (s3 FS lookups are expensive)


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java ac08ab3 
  ql/src/java/org/apache/hadoop/hive/ql/Context.java ec5d693 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java cadda8f 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 9d927bd 

Diff: https://reviews.apache.org/r/50434/diff/


Testing
-------


Thanks,

Rajesh Balamohan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message