impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roberto Cerioni, Paulo" <>
Subject Impala Admission Control concurrent query runtime
Date Wed, 28 Feb 2018 19:37:30 GMT

I've done some tests with concurrent queries running in the same queue and it seems the runtime
always receive a penalty compared to a single query running, even if we have plenty of resources
available to run multiple queries concurrently in that queue. In addition, I tried that by
running the same query concurrently in different queues and the runtime received a penalty
as well. For instance, if a query runtime running alone in the cluster is, let's say, 10 seconds,
it seems that I can only achieve that performance when no other query is running in the cluster,
otherwise there is a penalty of around 50% in runtime.

Other behavior I noticed was that the result of queries is being released in "batches" according
with the concurrency I have set (that might even be the reason for the aforementioned issue).
For example, if I run a single query, I get the answer in 1 second, but if I run 2 of that
same query concurrently the results of both queries will come after 1.5s. If I run 5 of the
same query concurrently, then the result of all 5 queries is going to come back at the same
time after 3 seconds, and so on. The penalty in runtime always increases with the number of
queries I run concurrently, because the first issued query apparently has to wait the whole
"batch" to execute.

Could you please explain what is happening under the hood that might be driving this behavior?
Is this supposed to work this way in the latest version of Impala we have or is there something
in the configuration I can use to prevent this? Is that planned for Impala's roadmap?



View raw message