drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-4591) Extend config system with distrib, site, node property files
Date Tue, 12 Apr 2016 19:01:25 GMT

     [ https://issues.apache.org/jira/browse/DRILL-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Paul Rogers updated DRILL-4591:
    Priority: Major  (was: Minor)

> Extend config system with distrib, site, node property files
> ------------------------------------------------------------
>                 Key: DRILL-4591
>                 URL: https://issues.apache.org/jira/browse/DRILL-4591
>             Project: Apache Drill
>          Issue Type: Improvement
>            Reporter: Paul Rogers
> Today Drill provides the drill-override.conf file to set Drill properties, and the drill-env.sh
file to provide custom launch properties. Today, most users seem to have a copy of DRILL_HOME
per node, and thus they copy these two files per-node. The result is that the two files act
as both the overall "site" configuration (for all nodes) and the "per-node" configuration
for that one node.
> In addition, some distributions of Drill (such as MapR), modify the "user" config files
with settings for that distribution. Now, the same files hold settings for the distribution,
site and node.
> The approach works, but is awkward. Ideally, provide the option to have three sets of
files: for the distribution, site, and node.
> Let's assume each set resides in its own directory. (Merging the three sets into a single
directory would simply shift the sync problems from files to directories.)
> * DRILL_HOME/conf: Distribution files
> * DRILL_SITE_DIR/conf: Site-wide files
> * DRILL_NODE_DIR/conf: Node-specific files
> Each directory might contain its own drill-override.conf, drill-env.sh files. (The idea
extends to site jar files as well by adding a jars directory under DRILL_SITE_DIR.)
> Configuration now provides additional levels:
> * Drill defaults (drill default and module conf files, code in drill-config.sh)
> * Distribution settings (special JVM settings, say)
> * Site settings (standard log or spill file locations)
> * Node settings
> * Launch settings (environment variables, -Dname=value options)
> The improvement becomes more important if a user employs NFS, MapR FS or YARN to automatically
deploy the site-wide files. In that case, the site files cannot also act as per-node files.
> The improvement also simplifies upgrades. Today, users must copy customizations from
and old to a new install. With the new system, Drill files are complely separated from user
files, making upgrades (of software) trivial.
> Note that the current version of Drill does allow users to put config files in /etc/drill/conf,
but that location replaces $DRILL_HOME/conf; the user must still start with the Distribution-specific
files, and must merge any new distribution changes in each new release.
> For backward compatibility, the site and node directories are optional and ignored if
the environment variables are not set. The site and node config files should be optional:
skip them if they do not exist (or, for node files, skip them if DRILL_NODE_CONF_DIR is not

This message was sent by Atlassian JIRA

View raw message