impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Robinson (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-3882: Simplify some query exec state locking
Date Thu, 03 Nov 2016 22:55:58 GMT
Henry Robinson has uploaded a new change for review.

Change subject: IMPALA-3882: Simplify some query exec state locking

IMPALA-3882: Simplify some query exec state locking

query_exec_state_map_lock_ is implicated in many deadlock paths, and is
heavily contended. This patch simplifies the usage of
query_exec_state_map_lock_ by requiring that it is a 'terminal' lock -
i.e. that it is incorrect to take any other locks while holding the map

The most significant change is to remove the code path from
GetQueryExecState() that could take QueryExecState::lock() *before*
yielding the query_exec_state_map_lock_. Instead, callers have to take
the exec state lock themselves.

Also change registration path so that QueryExecState::lock is not
held during planning. This prevents blocking e.g. web pages that need to
read QES data structures to produce their output.

A subtlety of the previous registration code path is that it forced
callers of get_metadata() rpcs to wait until registration and planning
was complete so that the query metadata was always available. This patch
replaces that logic with a promise that the RPC can block on, which is
triggered when metadata is available, rather than implicitly relying on
the locking protocol.

Also rename some QES methods to make it clearer what they do.

Change-Id: I516357d2b5e9eb83e8209872cbe4c078c778a629
M be/src/service/
M be/src/service/
M be/src/service/
M be/src/service/
M be/src/service/impala-server.h
M be/src/service/
M be/src/service/query-exec-state.h
7 files changed, 139 insertions(+), 146 deletions(-)

  git pull ssh:// refs/changes/35/4935/3
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newchange
Gerrit-Change-Id: I516357d2b5e9eb83e8209872cbe4c078c778a629
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <>

View raw message