impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Vissapragada (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock contention
Date Tue, 16 May 2017 02:02:39 GMT
Bharath Vissapragada has uploaded a new patch set (#6).

Change subject: IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock_ contention
......................................................................

IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock_ contention

Holding query_exec_state_map_lock_ and QES::lock_ together in certain
paths could potentially block the impalad from registering new queries.
The most common occurrence of this is while loading the webpage of a
query while the query planning is still in progress. Since we hold the
QES::lock_ during planning, it blocks the web page from loading which
inturn blocks incoming queries by holding query_exec_state_map_lock_.

While the actual fix for this is to not hold the QES::lock_ during
metadata load, this patch doesn't do it since it could have some unknown
side effects. Instead,

- We make sure that QES::lock_ doesn't block query_exec_state_map_lock_
  in hot paths.
- As an exception, we don't grab QES::lock_ while the query planning is
  in progress.

Also, this patch removes locking inside the GetQueryExecState() method.
It is confusing that some callers choose to lock it inside the method
and wait on it later. Now, the locking is the responsibility of the
callers.

Testing: Tested it locally by adding a long sleep in
JniFrontend.createExecRequest() and still was able to refresh the web UI
and run parallel queries.

Change-Id: Ie44daa93e3ae4d04d091261f3ec4891caffe8026
---
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
A tests/custom_cluster/test_query_concurrency.py
M www/query_plan.tmpl
7 files changed, 144 insertions(+), 37 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/6707/6
-- 
To view, visit http://gerrit.cloudera.org:8080/6707
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie44daa93e3ae4d04d091261f3ec4891caffe8026
Gerrit-PatchSet: 6
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>

Mime
View raw message