impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Hecht (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock contention
Date Tue, 16 May 2017 22:48:09 GMT
Dan Hecht has posted comments on this change.

Change subject: IMPALA-1972/IMPALA-3882: Fix QueryExecState::lock_ contention
......................................................................


Patch Set 6:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/6707/6/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:

Line 192:   DEFINE_int64(metadata_loading_pause_injection_ms, 0, "Simulates metadata loading
for a "
how about prefixing the flag with "stress_", like we do for the one in free-pool.h, to give
more of a clue that this is just for stressing the system.


PS6, Line 822:     // Note: this acquires the exec_state lock *before* the
             :     // query_exec_state_map_ lock. This is the opposite of
             :     // GetQueryExecState(..., true), and therefore looks like a
             :     // candidate for deadlock. The reason this works here is that
             :     // GetQueryExecState cannot find exec_state (under the exec state
             :     // map lock) and take it's lock until RegisterQuery has
             :     // finished. By that point, the exec state map lock will have been
             :     // given up, so the classic deadlock interleaving is not possible.
this comment is outdated now.


http://gerrit.cloudera.org:8080/#/c/6707/6/be/src/service/impala-server.h
File be/src/service/impala-server.h:

PS6, Line 85: /// 4. query_exec_state_map_lock_
i think this should be updated. with this change, do we ever hold this lock in conjunction
with another lock?


http://gerrit.cloudera.org:8080/#/c/6707/6/tests/custom_cluster/test_query_concurrency.py
File tests/custom_cluster/test_query_concurrency.py:

PS6, Line 55: GetRuntimeProfileStr
This doesn't make sense. GetRuntimeProfileStr() doesn't hold the map lock while getting the
QES::lock_.  I think what's happening is that the call on line 76 to get_in_flight_queries()
is hitting the QuerySummaryHandler() handler (please double check that), which then causes
the lock cycle. 

Without the fix, does the test ever get past line 76? I would expect the webserver to block
on the QES::lock_ at that point.


-- 
To view, visit http://gerrit.cloudera.org:8080/6707
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie44daa93e3ae4d04d091261f3ec4891caffe8026
Gerrit-PatchSet: 6
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message