hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chengxiang li" <chengxiang...@intel.com>
Subject Re: Review Request 27987: HIVE-8833 implement remote spark client
Date Tue, 18 Nov 2014 01:37:04 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27987/
-----------------------------------------------------------

(Updated Nov. 18, 2014, 1:37 a.m.)


Review request for hive, Rui Li, Szehon Ho, and Xuefu Zhang.


Changes
-------

fixed szehon mentioned issues.


Bugs: HIVE-8833
    https://issues.apache.org/jira/browse/HIVE-8833


Repository: hive-git


Description
-------

Hive would support submitting spark job through both local spark client and remote spark client.
we should unify the spark client API, and implement remote spark client through Remote Spark
Context.


Diffs (updated)
-----

  ql/pom.xml 076f993 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClient.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkClient.java 3cca7f4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 2fea62d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java e3e6d16 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java 51e0510 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobRef.java bf43b6e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SetSparkReducerParallelism.java d4d14a3

  spark-client/src/main/java/org/apache/hive/spark/client/SparkClient.java 8346b28 
  spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 5af66ee 

Diff: https://reviews.apache.org/r/27987/diff/


Testing
-------


Thanks,

chengxiang li


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message