impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@apache.org>
Subject Upcoming RPC changes - sequencing of patches
Date Tue, 10 Jan 2017 00:51:46 GMT
Hi -

I've been making a lot of good progress on IMPALA-2567 - replacing our
Thrift RPC stack with one based on Apache Kudu's. You can see the progress
I'm making on my personal branch here:

http://github.com/HenryR/Impala/tree/krpc

It's time to start merging some of these changes to master, to prevent
divergence and to make some of the scalability benefits available to all.
Here's my thoughts on the sequencing:

1. Commit thirdparty dependency changes that bring in protobuf, crcutil and
libev

2. Commit Kudu dependencies (including gutil upgrade). They will be unused,
but will compile as part of the usual build.

3. Commit change to add new RpcMgr abstraction, and port of Statestore
services to KRPC. We hope to have security finished by this point so as not
to disable security in trunk at any time.

4. Commit port of catalog service to KRPC.

5. Commit port of ImpalaInternalService to KRPC. This is a large change, as
some of the query execution paths change significantly now that we don't
need application-level thread pools etc.

These steps can happen concurrently, but that's the order I want to start
them in. I plan to submit the changes for step 1 very soon, and step 2 (at
least) later this week. The Impala-side changes remain a work in progress -
particularly the ImpalaInternalService ones - but have solidified enough,
and have undergone sufficient testing, that I think they are nearing
readiness for review.

Let me know if you have questions!

Cheers,
Henry

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message