drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael England (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3675) Node-label style drillbit ringfencing for Drill queues for true multi-tenancy
Date Thu, 20 Aug 2015 14:03:46 GMT
Michael England created DRILL-3675:
--------------------------------------

             Summary: Node-label style drillbit ringfencing for Drill queues for true multi-tenancy
                 Key: DRILL-3675
                 URL: https://issues.apache.org/jira/browse/DRILL-3675
             Project: Apache Drill
          Issue Type: Improvement
            Reporter: Michael England


Many of us are used to node labels in YARN which allows jobs to be submitted to a queue which
only executes on a subset of nodes e.g. in a 100 node cluster, queue 1 may only be allowed
to execute jobs on nodes 1-20, queue 2 can execute on nodes 21-50 and so on. This is necessary
in a true multi-tenant environment.

Drill should have the ability to set up queues with node/drillbit labels which only allows
queries submitted to that queue to execute on a defined set of drillbits. This would allow
completely different applications to each have their own queue and run on a subset of nodes,
meaning that resource contention is avoided. Currently, if multiple applications share a cluster
and use drill, they can cause resource contention and blocking, meaning it is inadequate for
a production multi-tenant environment.

In many mid-sized companies, different application areas will purchase a subset of nodes to
be used as part of a large cluster, and therefore Drill should provide the option to limit
Drill queries to the nodes allocated to them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message