drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Khatua (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3896) Allow for dynamic allocation of resources during execution of concurrent queries
Date Mon, 05 Oct 2015 21:02:26 GMT
Kunal Khatua created DRILL-3896:
-----------------------------------

             Summary: Allow for dynamic allocation of resources during execution of concurrent
queries
                 Key: DRILL-3896
                 URL: https://issues.apache.org/jira/browse/DRILL-3896
             Project: Apache Drill
          Issue Type: Improvement
          Components: Query Planning & Optimization
            Reporter: Kunal Khatua


With increasing number of concurrent queries, we often have to tune the Drill session settings
like `planner.width.max_per_node` to reduce committing of resources. Most RDBMS systems dynamically
allocate resources as more concurrent queries are running on the system.

A proposal would be to have a flag that enable/disables this dynamic allocation, along with
a min/max of the resource. For e.g., on a 10-node setup, the first 4 queries can concurrently
run with `planner.width.max_per_node`=23 (default for a 32 core/node system), and every additional
4 queries can reduce this setting for those session by about 2.. down to a minimum value of
6.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message