impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@cloudera.com>
Subject Re: [Impala-ASF-CR] IMPALA-4670 / IMPALA-4672: Add RpcMgr and port Statestore services to KRPC
Date Thu, 26 Jan 2017 03:21:23 GMT
Hm, I originally tried to upload it as a draft, but it published the patch
e-mail to everyone (and I decided it was ok for review). Marcel and Sailesh
have successfully published comments on PS3, so it seems that it was
visible.

I just 'published' the draft. Hopefully that fixes any remaining
inconsistencies.

On 25 January 2017 at 19:18, Henry Robinson (Code Review) <
gerrit@cloudera.org> wrote:

> Henry Robinson has uploaded a new patch set (#3).
>
> Change subject: IMPALA-4670 / IMPALA-4672: Add RpcMgr and port Statestore
> services to KRPC
> ......................................................................
>
> IMPALA-4670 / IMPALA-4672: Add RpcMgr and port Statestore services to KRPC
>
> This patch adds the core abstractions necessary for using Kudu RPC for
> Impala's backend services. The StatestoreService and
> StatestoreSubscriberService are both ported to KRPC to demonstrate how
> the new RPC framework operates.
>
> The main new class is RpcMgr, which is responsible for both services run
> by a daemon and for obtaining clients to remote services.
>
> Also done:
>  * Utility code to convert from a Thrift structure to a Protobuf
>    wrapper. Thrift RPCs do not need to be ported to Protobuf to work.
>  * Added more build support for Protobuf generation. All library targets
>    now depend on the Protobuf generation step, just as they do Thrift.
>  * thrift-server-test is rewritten to use Beeswax as a test service,
>    since the old StatestoreService has been removed.
>
> TESTING:
>  * Impala's test suite passes.
>  * statestore-test has been rewritten to use new statestore code.
>  * rpc-mgr-test unit-tests the RpcMgr and Rpc classes.
>
> TODO:
>  * Enable SSL and Kerberos support (already in Kudu's imported
>    libraries)
>  * Rewrite test_statestore.py in C++. We don't have protobuf support in
>    Python.
>
> Change-Id: I8dbf12b9ecd71d26d239d31c19b487175194c766
> ---
> M be/generated-sources/gen-cpp/CMakeLists.txt
> M be/src/catalog/CMakeLists.txt
> M be/src/catalog/catalog-server.cc
> M be/src/catalog/catalog-server.h
> M be/src/codegen/CMakeLists.txt
> M be/src/common/CMakeLists.txt
> M be/src/exec/CMakeLists.txt
> M be/src/exec/catalog-op-executor.cc
> M be/src/exec/hdfs-avro-scanner-test.cc
> M be/src/exec/kudu-util.h
> M be/src/experiments/CMakeLists.txt
> M be/src/exprs/CMakeLists.txt
> M be/src/rpc/CMakeLists.txt
> A be/src/rpc/common.proto
> A be/src/rpc/rpc-mgr-test.cc
> A be/src/rpc/rpc-mgr.cc
> A be/src/rpc/rpc-mgr.h
> A be/src/rpc/rpc-mgr.inline.h
> A be/src/rpc/rpc.h
> A be/src/rpc/rpc_test.proto
> M be/src/rpc/thrift-server-test.cc
> M be/src/rpc/thrift-util.h
> M be/src/runtime/CMakeLists.txt
> M be/src/runtime/buffered-tuple-stream-test.cc
> M be/src/runtime/bufferpool/CMakeLists.txt
> M be/src/runtime/exec-env.cc
> M be/src/runtime/exec-env.h
> M be/src/scheduling/CMakeLists.txt
> M be/src/service/CMakeLists.txt
> M be/src/service/query-exec-state.cc
> M be/src/statestore/CMakeLists.txt
> M be/src/statestore/statestore-subscriber.cc
> M be/src/statestore/statestore-subscriber.h
> A be/src/statestore/statestore-test.cc
> M be/src/statestore/statestore.cc
> M be/src/statestore/statestore.h
> A be/src/statestore/statestore.proto
> M be/src/statestore/statestored-main.cc
> M be/src/testutil/CMakeLists.txt
> M be/src/transport/CMakeLists.txt
> M be/src/udf/CMakeLists.txt
> M be/src/udf_samples/CMakeLists.txt
> M be/src/util/CMakeLists.txt
> M be/src/util/collection-metrics.h
> M common/thrift/CMakeLists.txt
> M common/thrift/StatestoreService.thrift
> M tests/statestore/test_statestore.py
> 47 files changed, 1,771 insertions(+), 749 deletions(-)
>
>
>   git pull ssh://gerrit.cloudera.org:29418/Impala-ASF
> refs/changes/20/5720/3
> --
> To view, visit http://gerrit.cloudera.org:8080/5720
> To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
>
> Gerrit-MessageType: newpatchset
> Gerrit-Change-Id: I8dbf12b9ecd71d26d239d31c19b487175194c766
> Gerrit-PatchSet: 3
> Gerrit-Project: Impala-ASF
> Gerrit-Branch: master
> Gerrit-Owner: Henry Robinson <henry@cloudera.com>
> Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
> Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
> Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
> Gerrit-Reviewer: Juan Yu <jyu@cloudera.com>
> Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
> Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>
> Gerrit-Reviewer: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
>
> --
> You received this message because you are subscribed to the Google Groups
> "impala-cr" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to impala-cr+unsubscribe@cloudera.com.
> For more options, visit https://groups.google.com/a/cloudera.com/d/optout.
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message