hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <sze...@cloudera.com>
Subject Review Request 16938: HIVE-6209 'LOAD DATA INPATH ... OVERWRITE ..' doesn't overwrite current data
Date Thu, 16 Jan 2014 01:45:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16938/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-6209
    https://issues.apache.org/jira/browse/HIVE-6209


Repository: hive-git


Description
-------

There was a wrong condition introduced in HIVE-3756, that prevented load data overwrite from
working properly.  In these situations, destf == oldPath == /user/warehouse/hive/<tableName>,
so -rmr was skipped on old data.

Note that if file name was same, ie load data inpath '<path>' with same path repeatedly,
it would work as the rename would overwrite the old data file.  But in this case, the filename
is different.

Other minor changes are trying to improve logging in this area to better diagnose the issues
(for example file permission, etc).


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 2fe86e1 

Diff: https://reviews.apache.org/r/16938/diff/


Testing
-------

The primary concern was whether removing the directory in these scenarios would make the rename
fail.  It should not due to fs.mkdirs call before, but I still verified the following scenarios:

load/insert overwrite into table with partitions
load/insert overwrite into table with buckets


Thanks,

Szehon Ho


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message