drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elijah Zupancic (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5918) Allow manual configuration when using Runtime.getRuntime().availableProcessors()
Date Tue, 31 Oct 2017 19:23:02 GMT
Elijah Zupancic created DRILL-5918:
--------------------------------------

             Summary: Allow manual configuration when using Runtime.getRuntime().availableProcessors()
                 Key: DRILL-5918
                 URL: https://issues.apache.org/jira/browse/DRILL-5918
             Project: Apache Drill
          Issue Type: Improvement
    Affects Versions: 1.11.0
            Reporter: Elijah Zupancic
            Priority: Critical


Currently Drill auto-configures the number of threads in various thread pools based on the
processor count:

https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java#L110
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java#L269
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/rm/AbstractResourceManager.java#L55
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/rm/DefaultResourceManager.java#L99

In a number of situations, this configuration is incorrect. In particular, the settings aren't
correct when [running in a bare-metal container](https://docs.google.com/document/d/1WGPGiJtbJZPZBMSD9YHAAz-O4AlAET9qOnS_GeriZtE/edit?usp=sharing)
because you don't necessarily have access to all of the compute shares or memory of the underlying
host.

Ideally, the number of CPUs to use could be specified as a system parameter to Drill. This
would allow users to configure Drill to their optimal settings.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message