hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <>
Subject Re: Review Request 49881: HIVE-14204: Optimize loading loaddynamic partitions
Date Thu, 21 Jul 2016 10:15:39 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated July 21, 2016, 10:15 a.m.)

Review request for hive and Ashutosh Chauhan.

Bugs: HIVE-14204

Repository: hive-git


Lots of time is spent in sequential fashion to load dynamic partitioned dataset in driver

E.g simple dynamic partitioned load as follows takes 300+ seconds

INSERT INTO web_sales_test partition(ws_sold_date_sk) select * from tpcds_bin_partitioned_orc_200.web_sales;

Time taken to load dynamic partitions: 309.22 seconds

Diffs (updated)

  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/ b4ae1d1 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/ 7b0369d 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/ PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/metadata/ ef0bb3d 




Rajesh Balamohan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message