impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-3977: TransmitData() should not block
Date Thu, 15 Dec 2016 22:55:27 GMT
Sailesh Mukil has uploaded a new patch set (#3).

Change subject: IMPALA-3977: TransmitData() should not block
......................................................................

IMPALA-3977: TransmitData() should not block

TransmitData() calls effectively block until an error occurs (e.g.
datastream_sender_timeout_ms expires) or the RPC is serviced by the
receiver. This is bad for a variety of reasons, not least that it
makes it impossible to cancel the sender without also cancelling the
recipient (who might not even be running!).

With our current RPC stack, this holds up the connection for as long
as the receiver takes to get ready (or until it times out), which
means that that connection is unusable by other threads.

When we move to KuduRPC, we will have a different problem if this RPC
remains blocking, in that it will hold up a service thread on the
receiver side potentially blocking progress for other tasks on that
host that require a service thread.

This patch introduces a EAGAIN like error code that the receiver sends
back to the sender if the DataStreamReceiver object is not yet ready
to receive data. The sender tries again up to 5 times with intervals
in between.

Change-Id: I47f9d6c1de95a72ab73471dcd2a3118ef13e7db0
---
M be/src/runtime/data-stream-mgr.cc
M be/src/runtime/data-stream-mgr.h
M be/src/runtime/data-stream-sender.cc
M common/thrift/generate_error_codes.py
4 files changed, 86 insertions(+), 142 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/5491/3
-- 
To view, visit http://gerrit.cloudera.org:8080/5491
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I47f9d6c1de95a72ab73471dcd2a3118ef13e7db0
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>

Mime
View raw message