accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: client config files
Date Fri, 02 Aug 2013 20:02:13 GMT
Okay, so there is implementation overlap, but that overlap is pretty
minimal (admittedly, it could potentially grow). The only thing it is
currently used for, is to carry the value of the RPC timeout, and this
is not currently very friendly to end users (they'd have to
instantiate something that extends AccumuloConfiguration, just to
override that one property's value, and then call
instance.setConfiguration()).

I like configuration with scopes, so I would prefer to have
configuration properties scoped to RPC. I implemented the
Property-enum configuration model to reflect this preference, but I
made the mistake of carrying all the scopes in a single configuration
object. I don't see a problem separating out the RPC scope into a
separate configuration object that could be shared between the other,
broader scopes ("client", "master", "tserver", etc.). In other words,
I'd prefer smaller, more localized scopes for configuration, and I
think it's sensible to have an RPC/connection scope (not necessarily
thrift-specific). See BatchWriterConfig for a good example of a
locally-scoped configuration.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Fri, Aug 2, 2013 at 3:24 PM, Michael Berman <mberman@sqrrl.com> wrote:
> I believe it is an implementation overlap.  Both ZKInstance and the
> master-tablet thrift connections get created in ThriftUtil.getClient().
>  Higher up in the stack, in both paths, we have access to an Instance from
> which to draw configuration (with getConfiguration()).  In one case, it's a
> ZKInstance with a degenerate AccumuloConfiguration, and in the other case
> it's an HDFSInstance with a site.xml-backed configuration, but the thrift
> stack makes no distinction.  It seems silly to me to introduce a
> distinction all the way down the stack just so we can have two different
> config sources (which have many of the same flags).  Unless we were going
> to implement it as a ThriftConnectionConfiguration interface with named
> methods that both AccumuloConfiguration and ClientConfiguration could
> implement...but that would be a big departure from the Property enum
> configuration model.
>
>
> On Fri, Aug 2, 2013 at 2:29 PM, Joey Echeverria <joey@cloudera.com> wrote:
>
>> Yeah, I agree. Consistency with Hadoop here is probably not that valuable.
>>
>> -Joey
>>
>> On Fri, Aug 2, 2013 at 2:28 PM, Keith Turner <keith@deenlo.com> wrote:
>> > On Thu, Aug 1, 2013 at 4:33 PM, Joey Echeverria <joey@cloudera.com>
>> wrote:
>> >
>> >> I generally prefer properties files to XML, but there may be a argument
>> >> for reusing Hadoop's SSL configuration system which is XML based.
>> >>
>> >
>> > I also prefer prefer properties files over XML.   The only reason I can
>> > think that we might want to use XML is for consistency with Hadoop and
>> > Accumulo server side config.  But it does not seem like a very compelling
>> > reason, its not like it prop files are hard to use once you realize you
>> > need to use them.
>> >
>> >
>> >>
>> >>
>> >> -Joey
>> >> —
>> >> Sent from Mailbox for iPhone
>> >>
>> >> On Thu, Aug 1, 2013 at 3:08 PM, Christopher <ctubbsii@apache.org>
>> wrote:
>> >>
>> >> > ^ Another reason I like commons-configuration here is for
>> >> > property-interpolation with HierarchicalConfiguration.
>> >> > --
>> >> > Christopher L Tubbs II
>> >> > http://gravatar.com/ctubbsii
>> >> > On Thu, Aug 1, 2013 at 3:07 PM, Christopher <ctubbsii@apache.org>
>> wrote:
>> >> >> I absolutely DO think they should be combined in a properties file
>> >> >> located in $HOME/.accumulo/config
>> >> >> I absolutely DO NOT think this client configuration should be
>> >> >> exclusive to the shell, and I absolutely DO NOT think it should
be
>> >> >> XML.
>> >> >>
>> >> >> I would love to see all our clients/client code use
>> >> >> commons-configuration to hold properties from the properties file,
so
>> >> >> that only a --config parameter is needed (with reasonable defaults,
>> so
>> >> >> even that is not absolutely necessary). I also think that every
>> >> >> property that can exist in the file should be possible to override
on
>> >> >> the command-line. I personally prefer to use system properties,
using
>> >> >> commons-configuration's HierarchicalConfiguration, but jcommander
may
>> >> >> make it easier to do the same thing in a slightly different way.
>> >> >>
>> >> >> --
>> >> >> Christopher L Tubbs II
>> >> >> http://gravatar.com/ctubbsii
>> >> >>
>> >> >>
>> >> >> On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman <mberman@sqrrl.com>
>> >> wrote:
>> >> >>> As part of SSL, we need to introduce configuration so accumulo
>> clients
>> >> >>> (such as ZooKeeperInstance) can find trust stores.  It seems
like
>> this
>> >> has
>> >> >>> a lot in common with shell config files in ACCUMULO-1397. 
Do people
>> >> think
>> >> >>> these should be combined, or should the shell have its own
separate
>> >> config?
>> >> >>>  I was imagining a simple java .properties-style key=value
list.
>>  Does
>> >> this
>> >> >>> seem reasonable?  Or should the format be more like the xml
of the
>> site
>> >> >>> config?  I was also imagining looking through a list of files
that
>> >> would
>> >> >>> each override settings, perhaps in the following order (from
lowest
>> to
>> >> >>> highest priority):
>> >> >>>
>> >> >>> /etc/accumulo/client.conf
>> >> >>> $ACCUMULO_HOME/conf/client.conf
>> >> >>> $HOME/.accumulo/config
>> >> >>> --client-config command line switch for shell or explicit parameter
>> >> passed
>> >> >>> to ZooKeeperInstance
>> >> >>>
>> >> >>> Does this sound good to y'all?  Should the explicit switch/parameter
>> >> have
>> >> >>> per-property override semantics, or should it just be used
as the
>> >> exclusive
>> >> >>> source of properties if specified?
>> >> >>>
>> >> >>> Mike Drob, are you actively working on the shell side of this
>> already?
>> >>  I
>> >> >>> see that bug is assigned to you...
>> >> >>>
>> >> >>> Thanks,
>> >> >>> Michael
>> >>
>>
>>
>>
>> --
>> Joey Echeverria
>> Director, Federal FTS
>> Cloudera, Inc.
>>

Mime
View raw message