hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Dere <>
Subject Review Request 63544: HIVE-17963: Fix for HIVE-17113 can be improved for non-blobstore filesystems
Date Fri, 03 Nov 2017 19:36:17 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hive, Ashutosh Chauhan and Owen O'Malley.

Bugs: HIVE-17963

Repository: hive-git


For non-blobstore FS:
1) Move the temp directory to a new directory name, to prevent additional files from being
added by any runaway processes.
2) Run removeTempOrDuplicateFiles() on this renamed temp directory
3) Run renameOrMoveFiles() to move the renamed temp directory to the final location.

For blobstore FS:
1) Remove duplicates from tmpPath
2) Use moveSpecifiedFiles() to perform a file-by-file move of the de-duped files to specPath

Also remove hive.exec.move.files.from.source.dir setting - the above changes will always be


  common/src/java/org/apache/hadoop/hive/conf/ cbe4de5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ 1be7eab 
  ql/src/test/queries/clientpositive/insert_with_move_files_from_source_dir.q 0117755 
  ql/src/test/queries/clientpositive/skewjoin.q 9ad4b77 




Jason Dere

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message