hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chengxiang li" <chengxiang...@intel.com>
Subject Re: Review Request 30107: HIVE-9410, ClassNotFoundException occurs during hive query case execution with UDF defined[Spark Branch]
Date Fri, 23 Jan 2015 06:37:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30107/
-----------------------------------------------------------

(Updated Jan. 23, 2015, 6:37 a.m.)


Review request for hive and Xuefu Zhang.


Changes
-------

add more comments and fix what xuefu mentioned before.


Bugs: HIVE-9410
    https://issues.apache.org/jira/browse/HIVE-9410


Repository: hive-git


Description
-------

The RemoteDriver does not contains added jar in it's classpath, so it would failed to desrialize
SparkWork due to NoClassFoundException. For Hive on MR, while use add jar through Hive CLI,
Hive add jar into CLI classpath(through thread context classloader) and add it to distributed
cache as well. Compare to Hive on MR, Hive on Spark has an extra RemoteDriver componnet, we
should add added jar into it's classpath as well.


Diffs (updated)
-----

  itests/src/test/resources/testconfiguration.properties 6340d1c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 9d9f4e6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java a4a166a 
  ql/src/test/queries/clientpositive/lateral_view_explode2.q PRE-CREATION 
  ql/src/test/results/clientpositive/lateral_view_explode2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/spark/lateral_view_explode2.q.out PRE-CREATION 
  spark-client/src/main/java/org/apache/hive/spark/client/JobContext.java 00aa4ec 
  spark-client/src/main/java/org/apache/hive/spark/client/JobContextImpl.java 1eb3ff2 
  spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 5f9be65 
  spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java PRE-CREATION


Diff: https://reviews.apache.org/r/30107/diff/


Testing
-------


Thanks,

chengxiang li


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message