pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nandor Kollar <nkol...@cloudera.com>
Subject Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0
Date Thu, 25 May 2017 07:13:46 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59530/
-----------------------------------------------------------

(Updated May 25, 2017, 7:13 a.m.)


Review request for pig, liyun zhang, Rohini Palaniswamy, and Adam Szita.


Repository: pig-git


Description
-------

Upgrade to Spark 2.1 API using shims.


Diffs (updated)
-----

  build.xml 4040fce 
  ivy/spark2.properties PRE-CREATION 
  shims/src/spark1/org/apache/pig/backend/hadoop/executionengine/spark/JobMetricsListener.java
PRE-CREATION 
  shims/src/spark1/org/apache/pig/backend/hadoop/executionengine/spark/converter/AbstractFlatMapFunction.java
PRE-CREATION 
  shims/src/spark1/org/apache/pig/backend/hadoop/executionengine/spark/converter/AbstractLimitConverter.java
PRE-CREATION 
  shims/src/spark1/org/apache/pig/backend/hadoop/executionengine/spark/converter/AbstractPairFlatMapFunction.java
PRE-CREATION 
  shims/src/spark1/org/apache/pig/backend/hadoop/executionengine/spark/converter/SkewedJoinUtil.java
PRE-CREATION 
  shims/src/spark1/org/apache/pig/tools/pigstats/spark/SparkJobStats.java PRE-CREATION 
  shims/src/spark2/org/apache/pig/backend/hadoop/executionengine/spark/JobMetricsListener.java
PRE-CREATION 
  shims/src/spark2/org/apache/pig/backend/hadoop/executionengine/spark/converter/AbstractFlatMapFunction.java
PRE-CREATION 
  shims/src/spark2/org/apache/pig/backend/hadoop/executionengine/spark/converter/AbstractLimitConverter.java
PRE-CREATION 
  shims/src/spark2/org/apache/pig/backend/hadoop/executionengine/spark/converter/AbstractPairFlatMapFunction.java
PRE-CREATION 
  shims/src/spark2/org/apache/pig/backend/hadoop/executionengine/spark/converter/SkewedJoinUtil.java
PRE-CREATION 
  shims/src/spark2/org/apache/pig/tools/pigstats/spark/SparkJobStats.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/spark/JobMetricsListener.java f813412

  src/org/apache/pig/backend/hadoop/executionengine/spark/SparkLauncher.java c6351e0 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/CollectedGroupConverter.java
83311df 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/FRJoinConverter.java 382258e

  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/ForEachConverter.java
b58415e 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/GlobalRearrangeConverter.java
f571b80 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/LimitConverter.java fe1b54c

  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/MergeCogroupConverter.java
adf78ec 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/MergeJoinConverter.java
d1c43b1 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/PoissonSampleConverter.java
e003bbd 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/SecondaryKeySortUtil.java
00d29b4 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/SkewedJoinConverter.java
c55ba31 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/SortConverter.java baabfa0

  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/SparkSampleSortConverter.java
3166fdc 
  src/org/apache/pig/backend/hadoop/executionengine/spark/converter/StreamConverter.java 3a50d48

  src/org/apache/pig/tools/pigstats/spark/SparkJobStats.java c8cc031 


Diff: https://reviews.apache.org/r/59530/diff/2/

Changes: https://reviews.apache.org/r/59530/diff/1-2/


Testing
-------


Thanks,

Nandor Kollar


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message