drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5547) Drill config options and session options do not work as intended
Date Tue, 11 Jul 2017 23:55:01 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083196#comment-16083196

ASF GitHub Bot commented on DRILL-5547:

Github user paul-rogers commented on a diff in the pull request:

    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java
    @@ -72,7 +72,16 @@ public SimpleParallelizer(QueryContext context) {
         OptionManager optionManager = context.getOptions();
         long sliceTarget = optionManager.getOption(ExecConstants.SLICE_TARGET).num_val;
         this.parallelizationThreshold = sliceTarget > 0 ? sliceTarget : 1;
    -    this.maxWidthPerNode = optionManager.getOption(ExecConstants.MAX_WIDTH_PER_NODE_KEY).num_val.intValue();
    +    // If the max_width is not set in the validator compute from the cpu load average
    +    long maxWidth = optionManager.getOption(ExecConstants.MAX_WIDTH_PER_NODE);
    +    if(maxWidth == 0) {
    +      Double cpu_load_average = optionManager.getOption(ExecConstants.CPU_LOAD_AVERAGE);
    +      this.maxWidthPerNode = (int)  Math.ceil(Runtime.getRuntime().availableProcessors()
* cpu_load_average);
    --- End diff --
    Probably should:
    * round result, not take the ceiling
    * clamp the number at 1 and the number of processors
    Math.max(1, Math.min( availProc, Math.round(... * ...) ) )
    We didn't do this before, but we should have...
    This is true if the value was set by a property, so the clamping should be done regardless
of whether the value is retrieved or computed. (Gracefully handle the case where someone sets
the value to 0 or 1 million, say.)

> Drill config options and session options do not work as intended
> ----------------------------------------------------------------
>                 Key: DRILL-5547
>                 URL: https://issues.apache.org/jira/browse/DRILL-5547
>             Project: Apache Drill
>          Issue Type: Bug
>          Components:  Server
>    Affects Versions: 1.10.0
>            Reporter: Karthikeyan Manivannan
>            Assignee: Venkata Jyothsna Donapati
>             Fix For: Future
> In Drill, session options should take precedence over config options. But several of
these session options are assigned hard-coded default values when the option validators are
initialized. Because of this config options will never be read and honored even if the user
did not specify the session option. 
> ClassCompilerSelector.JAVA_COMPILER_VALIDATOR uses CompilerPolicy.DEFAULT as the default
value. This default value gets into the session options map via the initialization of validators
in SystemOptionManager. 
> Now any piece of code that tries to check if a session option is set will never see a
null, so it will always use that value and never try to look into the config options. For
example, in the following piece of code from ClassCompilerSelector (), the policy will never
be read from the config file.
> policy = CompilerPolicy.valueOf((value != null) ? value.string_val.toUpperCase() : config.getString(JAVA_COMPILER_CONFIG).toUpperCase());

This message was sent by Atlassian JIRA

View raw message