hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <>
Subject Review Request 16938: HIVE-6209 'LOAD DATA INPATH ... OVERWRITE ..' doesn't overwrite current data
Date Thu, 16 Jan 2014 01:45:59 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hive.

Bugs: HIVE-6209

Repository: hive-git


There was a wrong condition introduced in HIVE-3756, that prevented load data overwrite from
working properly.  In these situations, destf == oldPath == /user/warehouse/hive/<tableName>,
so -rmr was skipped on old data.

Note that if file name was same, ie load data inpath '<path>' with same path repeatedly,
it would work as the rename would overwrite the old data file.  But in this case, the filename
is different.

Other minor changes are trying to improve logging in this area to better diagnose the issues
(for example file permission, etc).


  ql/src/java/org/apache/hadoop/hive/ql/metadata/ 2fe86e1 



The primary concern was whether removing the directory in these scenarios would make the rename
fail.  It should not due to fs.mkdirs call before, but I still verified the following scenarios:

load/insert overwrite into table with partitions
load/insert overwrite into table with buckets


Szehon Ho

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message