drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Farkas <tfar...@mapr.com>
Subject Re: Supporting Internal Options DRILL-5723
Date Thu, 17 Aug 2017 01:07:34 GMT
Hi All,


I've incorporated everyone's feedback and created a design doc. Let me know if you have any
comments.

https://docs.google.com/document/d/1Om3J0j6rV2Nk9zncSRQaJXPSWzYLHAdiVjbGWe6Adwo/edit?usp=sharing

Thanks,
Tim

________________________________
From: Parth Chandra <parthc@apache.org>
Sent: Tuesday, August 15, 2017 6:48:54 PM
To: dev
Subject: Re: Supporting Internal Options DRILL-5723

There is one case I can think of where this is useful - for testing a
server we may want to modify these parameters without  having to restart a
drillbit. With what I said above, that cannot be done.
Also, re security parameters: these can be exposed to show a user that they
are set, however their values are typically obscured, and not everyone can
change them. In other words these are options that are documented, have
restricted access but cannot be displayed.
Do we want to consider these? Obscuring the display is not hard, but
limiting access typically requires adding roles to users and that makes
this a different issue altogether.




On Tue, Aug 15, 2017 at 6:29 PM, Parth Chandra <parthc@apache.org> wrote:

> There are some examples of using internal (undocumented) configuration
> options in Drill. You simply don't put the option in drill-module.conf or
> in the SystemOptionManager. Adding any (arbitrary) key name in
> drill-override.conf automatically gives you an undocumented boot time
> option name. Or provide the options as a System property. See [1] as an
> example.
> This does show up in a query to sys.boot. However, if no one knows the
> name then it cannot be set (except by those that read the code, and nothing
> is hidden from them :) ), and it will not show up in the sys.boot either.
> The only case this is not safe is with security parameters, and even then
> it is a system administrator mistake to set an undocumented security
> parameter and leave it exposed.
> My $0.02
>
> [1] https://github.com/apache/drill/blob/master/exec/memory/
> base/src/main/java/org/apache/drill/exec/memory/BoundsChecking.java
>
>
>
> On Tue, Aug 15, 2017 at 5:09 PM, Paul Rogers <progers@mapr.com> wrote:
>
>> Hi Tim,
>>
>> As you noted, it might be easier to have a single options system (that
>> is, one for runtime system/session) options rather than two. If we have
>> two, we’d need to create two system tables, two session tables and two
>> bundles of options sent along with each query, and two sets of SQL ALTER
>> commands. We’d need two different internal APIs to get/set the options.
>> Doing so would rapidly become a fairly large project!
>>
>> On the other hand, if all we want to do is hide runtime options in the
>> web UI by default, we can do this using a metadata tag as part of the
>> option validator (defined in ExecConstants.java). In the iterator that
>> lists the options, we’d just skip any that are tagged as “internal” (say).
>>
>> Of course, if we did that, we’d need to add another table that lists the
>> internal runtime options. We’d need this if, say, user Fred has an issue
>> and posts to this list. Expert Barney says, “Hh, you need to change
>> such-and-so option.” How would Fred confirm he’d actually done so? He’d
>> need to query the internals options table.
>>
>> Another solution is simply to make the setting into a config (boot time)
>> option. If the setting is so obscure that users should never set it, is it
>> really worth having it in the system/session (runtime) option system?
>>
>> A different, but related, issue relates to config options. The config
>> system is hugely useful: we use it for user-set boot time options and to
>> externalize a large number of Drill constants. All that material shows up
>> if the user queries the boot options table using SQL. Do we want to filter
>> out the internal options from that system as well? If we do, maybe we can
>> use drill-override-example.conf for the filter: if an option appears in the
>> example file, then it is meant to be set by users and should appear in the
>> UI. If not, it is a private externalized constant that should not appear in
>> the UI.
>>
>> Thanks,
>>
>> - Paul
>>
>> > On Aug 15, 2017, at 4:34 PM, Timothy Farkas <tfarkas@mapr.com> wrote:
>> >
>> > Hello All,
>> >
>> > Boaz has proposed the idea of internal options. Currently all the
>> options are visible to to the user when they do:
>> >
>> > select * from sys.options.
>> >
>> > We would like internal options to be not visible to the user by
>> default. Possibly requiring them to do:
>> >
>> > select * from internal.options
>> >
>> > in order to see them, but internal options could be set in the same way
>> as other system options. The intention would be to add new options we
>> weren't comfortable with exposing to the end user as internal options.
>> After an internal option and its corresponding feature is considered stable
>> it could become a standard system option.
>> >
>> > This feature would be implemented on top of Jyosthna's feature
>> https://github.com/apache/drill/pull/868
>> >
>> > Paul has suggested this could be accomplished by adding metadata to
>> options in the SystemOptionManager, and distinguishing between standard an
>> internal options based on that meta data.
>> >
>> > If you have any feedback about how the feature should look like to the
>> user or be implemented. Please let us know.
>> >
>> > Thanks,
>> > Tim
>> >
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message