hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <rajesh.balamo...@gmail.com>
Subject Re: Review Request 49881: HIVE-14204: Optimize loading loaddynamic partitions
Date Thu, 21 Jul 2016 10:15:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49881/
-----------------------------------------------------------

(Updated July 21, 2016, 10:15 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-14204
    https://issues.apache.org/jira/browse/HIVE-14204


Repository: hive-git


Description
-------

Lots of time is spent in sequential fashion to load dynamic partitioned dataset in driver
side. 

E.g simple dynamic partitioned load as follows takes 300+ seconds

INSERT INTO web_sales_test partition(ws_sold_date_sk) select * from tpcds_bin_partitioned_orc_200.web_sales;

Time taken to load dynamic partitions: 309.22 seconds


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbLockManager.java b4ae1d1 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java 7b0369d 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/SynchronizedMetaStoreClient.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ef0bb3d 

Diff: https://reviews.apache.org/r/49881/diff/


Testing
-------


Thanks,

Rajesh Balamohan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message