impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4459: Consider making ReportExecStatus() RPC execute asynchronously
Date Mon, 14 Nov 2016 22:59:01 GMT
Sailesh Mukil has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/5078

Change subject: IMPALA-4459: Consider making ReportExecStatus() RPC execute asynchronously
......................................................................

IMPALA-4459: Consider making ReportExecStatus() RPC execute asynchronously

ReportExecStatus() has proved to be a long running RPC which blocks
the RPC sender for fairly long periods of time (in some extreme cases,
minutes). This change introduces a thread pool in the Coordinator
class that updates the exec status reports asynchronously.

The thread pool is intialized with only 1 thread currently as we do
not want a large number of threads if there are going to be a large
number of queries running on that coordinator.

At some point in the near future, we will also need to move to an
asynchronous reporting model such as this once we receive reports
per query per backend, versus how we do it now, i.e. per fragment
instance. A per query per backend report could potentially take a
long time to service and we shouldn't be doing it on a thrift
thread.

Change-Id: Ifb26bdfdcc96999b934435af32f5c32181a38c59
---
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
2 files changed, 41 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/5078/1
-- 
To view, visit http://gerrit.cloudera.org:8080/5078
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifb26bdfdcc96999b934435af32f5c32181a38c59
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Sailesh Mukil <sailesh@cloudera.com>

Mime
View raw message