impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Vissapragada (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2347: Reuse metastore client connections in Catalog
Date Mon, 27 Jun 2016 13:31:33 GMT
Bharath Vissapragada has posted comments on this change.

Change subject: IMPALA-2347: Reuse metastore client connections in Catalog
......................................................................


Patch Set 4:

Did some basic profiling with Jmeter on my local machine. The test runs 200 parallel create
table actions. Firstly with no. of initial threads set to 5 has throughput of 162/minute and
with set to 10, its 252/minute

http://www.dumpt.com/img/viewer.php?file=ktyu6271asjnnnahr512.png (with 5)
http://www.dumpt.com/img/viewer.php?file=rhx0syhkpw1tbynksvsm.png (with 10)

Now I benchmarked with and without the patch, the throughput increased by ~8 queries/minute

http://www.dumpt.com/img/viewer.php?file=rsa6p6mwhng5i3svtcji.png (without patch)
http://www.dumpt.com/img/viewer.php?file=9lgwf5rfxu052crp8zzt.png (with patch)

I think this still requires more finer profiling. Basically in the above setup, Impala limits
the no. queries admitted by impalad to 64 at any point (--fe_service_threads) and I think
tuning this can give even better results. (or basically profile it by firing to a load balancer,
so that all the queries aren't fired to the same impalad)

-- 
To view, visit http://gerrit.cloudera.org:8080/3381
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I517c0e1efef2584cd8d34017b33574f2ad69bd52
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Huaisi Xu <hxu@cloudera.com>
Gerrit-HasComments: No

Mime
View raw message