hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland" <br...@cloudera.com>
Subject Re: Review Request 18200: HIVE-860 - Persistent distributed cache
Date Tue, 18 Feb 2014 21:53:51 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18200/
-----------------------------------------------------------

(Updated Feb. 18, 2014, 9:53 p.m.)


Review request for hive.


Changes
-------

Makes caching configurable.


Bugs: HIVE-860
    https://issues.apache.org/jira/browse/HIVE-860


Repository: hive-git


Description
-------

Caches auxiliary jars and remote runtime jars in /user/$user/.hiveJars by their sha1 hash.
This results in:

1) faster queries
2) less distributed cache churn
3) a smaller/cleaner hive-exec jar


Diffs (updated)
-----

  bin/hive 3bd949f 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a182cd7 
  conf/hive-default.xml.template 0d08aa2 
  packaging/src/main/assembly/bin.xml a97ef7d 
  ql/pom.xml 53d0b9e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/HiveAuxClasspathBuilder.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 288da8e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/JarCache.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 326654f 
  shims/aggregator/pom.xml 7aa8c4c 

Diff: https://reviews.apache.org/r/18200/diff/


Testing
-------

Tested manually on a cluster.


Thanks,

Brock Noland


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message