impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4456: Address scalability issues of qs map lock and client request state map lock
Date Wed, 25 Oct 2017 22:48:30 GMT
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/8363 )

Change subject: IMPALA-4456: Address scalability issues of qs_map_lock_ and client_request_state_map_lock_
......................................................................


Patch Set 4:

(4 comments)

> Feel free to ignore, but I'm curious if you tried out SpinLock too.
 > be/src/benchmarks/lock-benchmark.cc shows that it is somehow more
 > efficient than boost::mutex under contention.

Using SpinLock shows better performance in the general case. So, I've switched qs_map_lock_
to use spin locks. I haven't done the same for client_request_state_map_lock_, since the access
patterns may be different, and it would be best to confirm with a benchmark first before making
a decision. I can add a benchmark in a follow on patch.

http://gerrit.cloudera.org:8080/#/c/8363/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/8363/3//COMMIT_MSG@30
PS3, Line 30: sharded to a default of 4 buckets initally.
> nit: sharded
Done


http://gerrit.cloudera.org:8080/#/c/8363/3/be/src/runtime/query-exec-mgr.cc
File be/src/runtime/query-exec-mgr.cc:

http://gerrit.cloudera.org:8080/#/c/8363/3/be/src/runtime/query-exec-mgr.cc@84
PS3, Line 84: DEFINE_int32(log_mem_usage_interval, 0, "If non-zero, impalad will output memory
usage "
            :     "every log_mem_usage_interval'th fragment complet
> Would it make sense to write a helper so that this code could look like:
Yup, that would make more sense. I've added a ScopedQueryStateMap {} class that locks/unlocks
the appropriate map(bucket) on scope entry/exit respectively.

I added a similar class ScopedCRSMap in impala-server.h/cc.


http://gerrit.cloudera.org:8080/#/c/8363/3/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:

http://gerrit.cloudera.org:8080/#/c/8363/3/be/src/service/impala-http-handler.cc@244
PS3, Line 244:   stringstream ss;
> 4 should be the constant here, no?
Yup, changed it.


http://gerrit.cloudera.org:8080/#/c/8363/3/be/src/service/impala-server.h
File be/src/service/impala-server.h:

http://gerrit.cloudera.org:8080/#/c/8363/3/be/src/service/impala-server.h@562
PS3, Line 562:   /// ClientRequestState's deletion. Also writes the query profile to the profile
log
> nit: corresponding
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/8363
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I61089090e1095da45a8a64ed3ccc78bd310807f1
Gerrit-Change-Number: 8363
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <philip@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Oct 2017 22:48:30 +0000
Gerrit-HasComments: Yes

Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message