hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Xiang" <jxi...@cloudera.com>
Subject Review Request 31811: HIVE-8851 Broadcast files for small tables via SparkContext.addFile() and SparkFiles.get() [Spark Branch]
Date Fri, 06 Mar 2015 22:22:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31811/
-----------------------------------------------------------

Review request for hive and Xuefu Zhang.


Bugs: HIVE-8851
    https://issues.apache.org/jira/browse/HIVE-8851


Repository: hive-git


Description
-------

Instead of loading small tables from HDFS, we add them to Spark context. Later on, we load
them back from Spark context local folder.


Diffs
-----

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java 91f67b7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java 3fd7ea9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HashTableLoader.java 129e97b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClient.java 383eebe 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java 7e33a3f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java f1d65ea 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 1342afe 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java 91d83f3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSession.java 3d4b39b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java 603f1ca 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 69004dc

  spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 5de7c2e 

Diff: https://reviews.apache.org/r/31811/diff/


Testing
-------

Unit test


Thanks,

Jimmy Xiang


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message