drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5547) Drill config options and session options do not work as intended
Date Fri, 09 Jun 2017 20:59:18 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045021#comment-16045021

Paul Rogers commented on DRILL-5547:

Proposal: Link the two config systems to allow custom distributions, or special cases, to
set the default values for system/session options using the config system.

At present, the runtime system/session option system is distinct from the boot-time config
system. The defaults for system/session options are hard coded in {{ExecConstants.java}} (among
other places). The way to change system/session globally is to issue an ALTER SYSTEM command.

It also turns out that, in a number of areas, we have ad-hoc linkage between the two systems.
For example, for the code gen compiler, we have logic that (tries, but fails) to check if
the system option is set, and if not, use a config option.

Suppose a custom software distribution of Drill wants to ship with system option defaults
other than what Apache Drill defines. The only ways to do that today are:

# A custom build of Drill with new hard-coded defaults.
# The client sets the options on each connection via ALTER SESSION.
# The installer fires up Drill and issues an ALTER SYSTEM command.

Option 1 works, but it requires  a custom build and cherry-picked changes going forward. Works,
but less than ideal.

For option 2, the client of the distribution (potentially many!) would have to alter the session
options in a way that that the distribution needs. This is not terribly reliable.

Option 3 requires the distribution installer to start Drill, issue ALTER SYSTEM options, and
shut down Drill. It works, but is awkward.

Let’s go back to how system options work. They are in three layers:

* Session options
* System options
* Hard-coded defaults in {{ExecConstants.java}} (and other places)

So, as we said, one way to create query service is to alter the code in {{ExecConstants}}
to have new defaults. 

Maybe we can do what UI tools often do for strings: externalize them in a Java properties
file. They are still defaults defined by Drill (not the user), but they are defined in a properties
file rather than in Java source code. Distributions of Drill are free to alter the defaults
(just as localizations alter user-visible strings in a UI tool.)

But, wait, Drill already has such a system. Instead of using properties files, why not use
the HOCON TypeSafe system we already have? Another JIRA is introducing a new {{drill-distrib.conf}}
file that sits between the existing Drill-provide {{drill-module.conf}} and the user-provided
{{drill-override.conf}} file. This file is parallel to the {{distrib-env.sh}} file introduced
in 1.8.

The key is to realize is that *users* won’t alter these defaults. Only the installer for
each custom Drill distribution, via {{drill-distrib.conf}}.

Could a use change the options in their {{drill-module.conf}} file, change it again via ALTER
SYSTEM, and get confused? Probably. But the solution is easy: don't do that.

The alternative is to leave the defaults hard-coded (with the issues noted above), or externalize
them into Java properties files, then implement parsers for the values (which, as it turns
out, we already have in the form of validators.)

Thoughts? Comments? Suggestions?

> Drill config options and session options do not work as intended
> ----------------------------------------------------------------
>                 Key: DRILL-5547
>                 URL: https://issues.apache.org/jira/browse/DRILL-5547
>             Project: Apache Drill
>          Issue Type: Bug
>          Components:  Server
>    Affects Versions: 1.10.0
>            Reporter: Karthikeyan Manivannan
>            Assignee: Venkata Jyothsna Donapati
>             Fix For: Future
> In Drill, session options should take precedence over config options. But several of
these session options are assigned hard-coded default values when the option validators are
initialized. Because of this config options will never be read and honored even if the user
did not specify the session option. 
> ClassCompilerSelector.JAVA_COMPILER_VALIDATOR uses CompilerPolicy.DEFAULT as the default
value. This default value gets into the session options map via the initialization of validators
in SystemOptionManager. 
> Now any piece of code that tries to check if a session option is set will never see a
null, so it will always use that value and never try to look into the config options. For
example, in the following piece of code from ClassCompilerSelector (), the policy will never
be read from the config file.
> policy = CompilerPolicy.valueOf((value != null) ? value.string_val.toUpperCase() : config.getString(JAVA_COMPILER_CONFIG).toUpperCase());

This message was sent by Atlassian JIRA

View raw message