incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: Default configuration parameters?
Date Mon, 09 Jan 2012 10:37:04 GMT
On 1/7/12 7:48 PM, Karthik Kambatla wrote:
> It definitely makes sense to have only Guice modules (one default and one
> app/user-specific that extends the default). What do you think of the
> following approach?
>
> DefaultModule {
>     -- list of protected variables
>
>     final configure() {
>          bind(parameter).to(variable)
>     }
> }
>
> UserSpecificModule extends DefaultModule {
>     override protected variables
> }
>
> This way the user can only modify existing parameters.


That's an interesting approach!

By documenting/annotating those protected variables, we would even have 
a simple way to keep an up-to-date list of available parameters with 
their documentation.

2 questions though:

- I wonder though how to be able to specify tunable parameters without 
recompiling? Or maybe we just provide a facility for that?
Or we could generate a Guice module file dynamically based on parameters 
from a config file (that way we get the proper checks automatically)?

- what about parameters for user applications? How to define and load 
parameters that are specific to user applications?


Also, instead of "hiding" parameters from users (we'll always need to 
tune some of them), we could provide an easy way to list them and their 
default values. (a script for instance: "s4 showConfig <appPath>" or "s4 
showConfig <S4R file>").



Matthieu

>
> Thanks
> Karthik
>
> On Sat, Jan 7, 2012 at 1:19 PM, Leo Neumeyer<leoneumeyer@gmail.com>  wrote:
>
>> I don't like the idea of having to have properties in two places
>> because it is hard to maintain and is error prone. If you mistype,
>> then the property will use the default value and the error will go
>> undetected. I dont' see why this is better than having defaults in
>> Guice Module. You would be adding another file that needs to be
>> maintained. We may as well hide the default configuration from the
>> user, no? If the site admin wants to make a change in the default
>> values he can change the Module (this would be a a rare change.)
>>
>> I would follow these principles:
>>
>> - Minimize config options.
>> - Provide default config that always runs out of the box.
>> - Have an authoritative list of properties in a single place to avoid
>> inconsistencies.
>>
>> Given this, I would use the current approach properties + Guice or
>> Guice only. Any other ideas?
>>
>> -leo
>>
>> On Sat, Jan 7, 2012 at 10:03 AM, Karthik Kambatla
>> <kkambatl@cs.purdue.edu>  wrote:
>>> How about having two property files - one which loads the default values,
>>> which will be subsequently overwritten by the user specified properties
>>> file, both loaded by the Guice modules sequentially one after the other?
>>>
>>> I have used xml version of similar property files (while using Hadoop)
>> and
>>> liked it.
>>>
>>> Thanks
>>> Karthik
>>>
>>> On Sat, Jan 7, 2012 at 12:49 PM, Leo Neumeyer<leoneumeyer@gmail.com>
>> wrote:
>>>
>>>> We should definitely have default values to run out of the box. There
>>>> should be zero friction to use the system after installation.
>>>>
>>>> Maybe it is a good idea to set all the properties in the Guice Module
>>>> and use the properties file to override them. In that way, we can ship
>>>> with a default Module that runs something like a cluster with 2 nodes
>>>> on the local host. We should still include the properties file with
>>>> the properties commented out. One problem is that we need to maintain
>>>> properties in two places (properties file and Guice Module) which is
>>>> error prone and cannot be verified in unit tests. BTW, the properties
>>>> class has methods to read properties and use defaults if the property
>>>> doesn't exist. The problem is the typos that will result in errors.
>>>>
>>>> The other option is to not have properties file and define everything
>>>> in the Guice Module. This solves the inconsistency but requires code
>>>> changes to change props. On the other hand I see the properties file
>>>> as another source file, that's the whole point of using Guice: not to
>>>> be afraid to use Java static typing for configuration.
>>>>
>>>> So basically, I am not answering your question :-)
>>>>
>>>> -leo
>>>>
>>>> On Fri, Jan 6, 2012 at 10:18 PM, Karthik Kambatla
>>>> <kkambatl@cs.purdue.edu>  wrote:
>>>>> We have several configuration parameters in our code base, which we
>> are
>>>>> currently supplying through our modules. It would be nice if we set
>>>> default
>>>>> values for these parameters, so that the user need not go through
>> every
>>>>> gory detail before running their apps. How can we go about it?
>>>>>
>>>>> 1. Using Guice, set default values for these parameters in the code
>> base
>>>>> itself - don't know how to go about it yet.
>>>>> 2. I like the hadoop approach - where they have default-<>.xml
and
>>>>> site-<>.xml. Parameters specified in the site xml override those
>>>> mentioned
>>>>> in the default.
>>>>>
>>>>> Should we take any of these routes, or should we just leave it for
>> now?
>>>> If
>>>>> you guys think it is important enough, I can create a JIRA for the
>> same.
>>>>>
>>>>> Thanks
>>>>> Karthik
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Leo Neumeyer (@leoneu)
>>>>
>>
>>
>>
>> --
>>
>> Leo Neumeyer (@leoneu)
>>
>


Mime
View raw message