impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Robinson (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5388: Only retry RPC on lost connection in send call
Date Mon, 05 Jun 2017 19:23:56 GMT
Henry Robinson has posted comments on this change.

Change subject: IMPALA-5388: Only retry RPC on lost connection in send call

Patch Set 2:


Were you able to test this at all?

The idea behind retrying in DoRpc() was to insulate callers from having to detect and implement
re-open / retry loops themselves. Because the client-cache keeps connections around for a
long time, they would sometimes get closed by the remote end. That expanded to include the
logic around retrying the recv part of an RPC if the socket read timeout had expired, since
TransmitData() can block for a long time before responding. Arguably it might be better to
set the timeouts for the individual RPC (that's what happens in KRPC). 

Although I think this is brittle, I feel like it's relatively correct _if_ our assumption
that the socket-closed exceptions we detect only get thrown on the write path is accurate.
It seems to be the case for non-secure clusters; for Kerberos or TLS it's less clear (but
it seems that receiving the response only involves read operations). 

Can you see a situation with this patch where an RPC will be spuriously retried?
File be/src/runtime/client-cache.h:

PS2, Line 252: else if 
nit: avoid 'else' since previous block will return. Here and below.

PS2, Line 253: <F, Request, Response>
can the compiler not figure out the specialization from the arguments?

PS2, Line 262: Status(TErrorCode::RPC_GENERAL_ERROR, e.what());
> Please note that this will start returning "RPC: SSL resource temporarily u
I think it's ok - it's actually hard to think of many cases where retrying an RPC was a good
idea (cancellation, maybe profile reporting...). I'd rather get error messages than incorrect

PS2, Line 285: // If it's not timeout exception, then the connection is broken, stop retrying.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I176975f2aa521d5be8a40de51067b1497923d09b
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Ho <>
Gerrit-Reviewer: Alan Choi <>
Gerrit-Reviewer: Henry Robinson <>
Gerrit-Reviewer: Juan Yu <>
Gerrit-Reviewer: Michael Ho <>
Gerrit-Reviewer: Sailesh Mukil <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message