hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergio Pena <>
Subject Re: Review Request 53966: HIVE-15199: INSERT INTO data on S3 is replacing the old rows with the new ones
Date Mon, 21 Nov 2016 23:54:22 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Nov. 21, 2016, 11:54 p.m.)

Review request for hive.


Attach a new patch that do not call destFs.exists when HDFS is used. This is to avoid performance
penalties when HDFS Is used.

Only change is on Hive.mvFile():

      boolean isBlobStoragePath = BlobStorageUtils.isBlobStoragePath(conf, destDirPath);
      while ((isBlobStoragePath && destFs.exists(destFilePath)) || !destFs.rename(sourcePath,
destFilePath)) {
        destFilePath = createCopyFilePath(destDirPath, name, type, ++counter);

Bugs: HIVE-15199

Repository: hive-git


The patch helps execute repeated INSERT INTO statements on S3 tables when the scratch directory
is on S3.

Diffs (updated)

  common/src/java/org/apache/hadoop/hive/common/ 1d8c04160c35e48781b20f8e6e14760c19df9ca5

  itests/hive-blobstore/src/test/queries/clientpositive/insert_into.q 919ff7d9c7cb40062d68b876d6acbc8efb8a8cf1

  itests/hive-blobstore/src/test/results/clientpositive/insert_into.q.out c25d0c4eec6983b6869e2eba711b39ba91a4c6e0

  ql/src/java/org/apache/hadoop/hive/ql/metadata/ 61b8bd0ac40cffcd6dca0fc874940066bc0aeffe




Sergio Pena

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message