hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin" <vanzin+...@cloudera.com>
Subject Re: Review Request 28779: [spark-client] Netty-based RPC implementation.
Date Tue, 09 Dec 2014 19:08:42 GMT


> On Dec. 9, 2014, 7:05 p.m., Xuefu Zhang wrote:
> > pom.xml, line 152
> > <https://reviews.apache.org/r/28779/diff/7/?file=786238#file786238line152>
> >
> >     Is there a reason that we cannot keep 3.7.0? Upgrading a dep version usually
gives some headaches.

This version is not used anywhere in the Hive build. In fact, there is no version "3.7.0.Final"
of "io.netty" (that's for the old "org.jboss.netty" package).


- Marcelo


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28779/#review64411
-----------------------------------------------------------


On Dec. 9, 2014, 6:49 p.m., Marcelo Vanzin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28779/
> -----------------------------------------------------------
> 
> (Updated Dec. 9, 2014, 6:49 p.m.)
> 
> 
> Review request for hive, Brock Noland, chengxiang li, Szehon Ho, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-9036
>     https://issues.apache.org/jira/browse/HIVE-9036
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This patch replaces akka with a simple netty-based RPC layer. It doesn't add any features
on top of the existing spark-client API, which is unchanged (except for the need to add empty
constructors in some places).
> 
> With the new backend we can think about adding some nice features such as future listeners
(which were awkward with akka because of Scala), but those are left for a different time.
> 
> The full change set, with more detailed descriptions, can be seen here:
> https://github.com/vanzin/hive/commits/spark-client-netty
> 
> 
> Diffs
> -----
> 
>   pom.xml 630b10ce35032e4b2dee50ef3dfe5feb58223b78 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/RemoteSparkJobStatus.java
PRE-CREATION 
>   spark-client/pom.xml PRE-CREATION 
>   spark-client/src/main/java/org/apache/hive/spark/client/ClientUtils.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/Protocol.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/SparkClientFactory.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/metrics/InputMetrics.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/metrics/Metrics.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleReadMetrics.java
PRE-CREATION 
>   spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleWriteMetrics.java
PRE-CREATION 
>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/KryoMessageCodec.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/README.md PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/Rpc.java PRE-CREATION 
>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcConfiguration.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcDispatcher.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcException.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcServer.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/counter/SparkCounter.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/counter/SparkCounterGroup.java PRE-CREATION

>   spark-client/src/main/java/org/apache/hive/spark/counter/SparkCounters.java PRE-CREATION

>   spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java PRE-CREATION

>   spark-client/src/test/java/org/apache/hive/spark/client/rpc/TestKryoMessageCodec.java
PRE-CREATION 
>   spark-client/src/test/java/org/apache/hive/spark/client/rpc/TestRpc.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/28779/diff/
> 
> 
> Testing
> -------
> 
> spark-client unit tests, plus some qtests.
> 
> 
> Thanks,
> 
> Marcelo Vanzin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message