accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: client config files
Date Fri, 02 Aug 2013 18:29:41 GMT
The overlap is only a conceptual overlap, not an implementation one.
Servers use HdfsZooInstance, which reads the xml configuration file,
and read the instanceId out of HDFS. Clients have ZooKeeperInstance,
which requires user input and gets the instanceId from an a convenient
pointer in ZK. Even if this were changed slightly to get either the
alias or the instanceID directly from a client configuration file, it
still wouldn't be quite the same, because we don't expect clients to
use the xml configuration file at all.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Fri, Aug 2, 2013 at 1:25 PM, Michael Berman <mberman@sqrrl.com> wrote:
> Yeah, I was starting to think along those lines in ACCUMULO-1397 (a generic
> configuration library).  Although options about how to make connections to
> accumulo are needed by both clients and accumulo services themselves (for
> masters talking to tablets, for example).
>
>
> On Fri, Aug 2, 2013 at 1:20 PM, Christopher <ctubbsii@apache.org> wrote:
>
>> Yes, I think that would be provide the best user experience for client
>> code. I'm not too stuck on this point, though I do think it should be
>> independent of AccumuloConfiguration for reasons I mentioned on
>> ACCUMULO-1397.
>>
>> I've been playing with the idea of creating a generic typed
>> configuration library that extends commons-configuration to make it
>> easier to get the same value as AccumuloConfiguration's Property
>> enums, but without being so monolithic and Accumulo server-specific.
>> That common interface could form the basis of an
>> AccumuloClientConfiguration, and independently, an
>> AccumuloServerConfiguration. Do you think that would be useful for
>> your client configuration?
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Fri, Aug 2, 2013 at 10:11 AM, Michael Berman <mberman@sqrrl.com> wrote:
>> > So does this mean you'd rather have the config switches be called
>> something
>> > like javax.net.ssl.trustStore rather than general.server.ssl.trustStore
>> in
>> > the Accumulo Properties?  Our implementation of SSL will be provided by
>> the
>> > thrift connectors rather than us using JSSE directly, so we'll have to
>> > interpret them ourselves rather than JSSE doing it automatically.  Should
>> > we respect these flags if they're passed in through
>> System.getProperties()
>> > in addition to whatever the normal config flow ends up being?  Note that
>> we
>> > won't be able to respect many of the flags, since we're at Thrift's mercy
>> > for socket construction.
>> >
>> >
>> > On Thu, Aug 1, 2013 at 10:06 PM, Christopher <ctubbsii@apache.org>
>> wrote:
>> >
>> >> I'm generally not a fan of the way some standard Java things have been
>> >> reinvented in Hadoop. I've always seen JSSE SSL config as system
>> >> properties, optionally stored in a properties file. Even if Hadoop is
>> >> using an XML-based configuration for this purpose, I'd still steer
>> >> clear of it, for this reason.
>> >>
>> >> --
>> >> Christopher L Tubbs II
>> >> http://gravatar.com/ctubbsii
>> >>
>> >>
>> >> On Thu, Aug 1, 2013 at 4:33 PM, Joey Echeverria <joey@cloudera.com>
>> wrote:
>> >> > I generally prefer properties files to XML, but there may be a
>> argument
>> >> for reusing Hadoop's SSL configuration system which is XML based.
>> >> >
>> >> >
>> >> > -Joey
>> >> > —
>> >> > Sent from Mailbox for iPhone
>> >> >
>> >> > On Thu, Aug 1, 2013 at 3:08 PM, Christopher <ctubbsii@apache.org>
>> wrote:
>> >> >
>> >> >> ^ Another reason I like commons-configuration here is for
>> >> >> property-interpolation with HierarchicalConfiguration.
>> >> >> --
>> >> >> Christopher L Tubbs II
>> >> >> http://gravatar.com/ctubbsii
>> >> >> On Thu, Aug 1, 2013 at 3:07 PM, Christopher <ctubbsii@apache.org>
>> >> wrote:
>> >> >>> I absolutely DO think they should be combined in a properties
file
>> >> >>> located in $HOME/.accumulo/config
>> >> >>> I absolutely DO NOT think this client configuration should
be
>> >> >>> exclusive to the shell, and I absolutely DO NOT think it should
be
>> >> >>> XML.
>> >> >>>
>> >> >>> I would love to see all our clients/client code use
>> >> >>> commons-configuration to hold properties from the properties
file,
>> so
>> >> >>> that only a --config parameter is needed (with reasonable defaults,
>> so
>> >> >>> even that is not absolutely necessary). I also think that every
>> >> >>> property that can exist in the file should be possible to override
>> on
>> >> >>> the command-line. I personally prefer to use system properties,
>> using
>> >> >>> commons-configuration's HierarchicalConfiguration, but jcommander
>> may
>> >> >>> make it easier to do the same thing in a slightly different
way.
>> >> >>>
>> >> >>> --
>> >> >>> Christopher L Tubbs II
>> >> >>> http://gravatar.com/ctubbsii
>> >> >>>
>> >> >>>
>> >> >>> On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman <mberman@sqrrl.com>
>> >> wrote:
>> >> >>>> As part of SSL, we need to introduce configuration so accumulo
>> clients
>> >> >>>> (such as ZooKeeperInstance) can find trust stores.  It
seems like
>> >> this has
>> >> >>>> a lot in common with shell config files in ACCUMULO-1397.
 Do
>> people
>> >> think
>> >> >>>> these should be combined, or should the shell have its
own separate
>> >> config?
>> >> >>>>  I was imagining a simple java .properties-style key=value
list.
>> >>  Does this
>> >> >>>> seem reasonable?  Or should the format be more like the
xml of the
>> >> site
>> >> >>>> config?  I was also imagining looking through a list of
files that
>> >> would
>> >> >>>> each override settings, perhaps in the following order
(from
>> lowest to
>> >> >>>> highest priority):
>> >> >>>>
>> >> >>>> /etc/accumulo/client.conf
>> >> >>>> $ACCUMULO_HOME/conf/client.conf
>> >> >>>> $HOME/.accumulo/config
>> >> >>>> --client-config command line switch for shell or explicit
parameter
>> >> passed
>> >> >>>> to ZooKeeperInstance
>> >> >>>>
>> >> >>>> Does this sound good to y'all?  Should the explicit
>> switch/parameter
>> >> have
>> >> >>>> per-property override semantics, or should it just be used
as the
>> >> exclusive
>> >> >>>> source of properties if specified?
>> >> >>>>
>> >> >>>> Mike Drob, are you actively working on the shell side of
this
>> >> already?  I
>> >> >>>> see that bug is assigned to you...
>> >> >>>>
>> >> >>>> Thanks,
>> >> >>>> Michael
>> >>
>>

Mime
View raw message