hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Xiang" <jxi...@cloudera.com>
Subject Re: Review Request 33251: HIVE-10302 Cache small tables in memory [Spark Branch]
Date Thu, 23 Apr 2015 17:48:47 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33251/
-----------------------------------------------------------

(Updated April 23, 2015, 5:48 p.m.)


Review request for hive, Chao Sun, Szehon Ho, and Xuefu Zhang.


Bugs: HIVE-10302
    https://issues.apache.org/jira/browse/HIVE-10302


Repository: hive-git


Description
-------

Cached the small table containter so that mapjoin tasks can use it if the task is executed
on the same Spark executor.
The cache is released right before the next job after the mapjoin job is done.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HashTableLoader.java fe108c4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HivePairFlatMapFunction.java 2f137f9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java 3f240f5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java 72ab913 

Diff: https://reviews.apache.org/r/33251/diff/


Testing
-------

Ran several queries in live cluster. ptest pending.


Thanks,

Jimmy Xiang


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message