zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Zookeeper with SSL release date
Date Tue, 22 Mar 2016 04:39:26 GMT
On Mon, Mar 21, 2016 at 9:16 PM, Alexander Shraer <shralex@gmail.com> wrote:
> another thing - shouldn't things like setting quotas also be part of the
> admin API ? how does that
> work now ?
>

Hm, good point. There is no code level API afaik - the quotas are in
/zookeeper znodes iirc. So if you care about controlling access to the
quota settings you need to setup ACLs. I don't think we thought about
that back in the day.

Patrick

> Alex
>
> On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <shralex@gmail.com> wrote:
>
>> I don't think that getConfig should be an admin functionality. It is
>> essential for client-side re-balancing
>> that we implemented (all clients shoudl be able to detect configuration
>> changes via getConfig). It could
>> be hidden somewhat by defining higher-level re-balancing
>> policies (ZOOKEEPER-2016)
>> but there hasn't been enough progress on that. Perhaps instead getConfig
>> should be controlled
>> by a separate flag ?
>>
>> Alex
>>
>> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <phunt@apache.org> wrote:
>>
>>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <shralex@gmail.com>
>>> wrote:
>>> > Hi Patrick, Flavio,
>>> >
>>> > Since there seems to be consensus on this, I can add this flag, unless
>>> > someone else wants to. I assume that getConfig should still work
>>> regardless
>>> > of the flag ? is there a security concern with clients knowing the list
>>> of
>>> > servers?
>>> >
>>>
>>> We've always hidden that detail from users. We don't even expose which
>>> server you're connected to today. I remember Ben (and perhaps Flavio?)
>>> highlighting this as important to maintain although I'm not super
>>> familiar with the specifics on why. It made sense to me though from
>>> the perspective that we don't want clients to make assumptions that
>>> probably shouldn't.
>>>
>>> My thinking is that we should 1) add a config option to enable
>>> reconfig (off by default), 2) move reconfig specific functionality
>>> from ZooKeeper.java (including getconfig) into an "admin" package,
>>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
>>> when folks do want to enable reconfig and are also worried about auth.
>>> (e.g. turn on kerb)
>>>
>>> Again, I don't see any of this as a quality issue personally. As such
>>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
>>> were interested in doing such a release. Adjusting the API should be
>>> done before we move to "beta" though. Although that seems like a
>>> pretty mechanical (eclipse/idea) type refactoring?
>>>
>>> Patrick
>>>
>>> > Cheers,
>>> > Alex
>>> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <phunt@apache.org> wrote:
>>> >
>>> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fpj@apache.org>
>>> wrote:
>>> >> > I gotta say that I'm not super excited about this option, but for
>>> some
>>> >> reason I ended up carrying the flag. To recap, I just raised this
>>> option
>>> >> because it seems that there are folks interested in features in 3.5
>>> like
>>> >> SSL and not necessarily in reconfiguration. SSL is important and to
>>> take
>>> >> Kafka as an example, it sucks that we can't have a whole set up using
>>> SSL.
>>> >> For ZK, the real questions are:
>>> >> >
>>> >> > 1- how fast can we make 3.5 stable?
>>> >> > 2- would it be faster if we have a way of disabling reconfiguration?
>>> >> > 3- would enough users care about a stable 3.5 that has
>>> reconfiguration
>>> >> disabled?
>>> >> >
>>> >> > It is taking a long time to get 3.5 to beta. There has been some
good
>>> >> activity around 3.5.2 release, which is a great step, but it is unclear
>>> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta
>>> then.
>>> >> Frankly, disabling reconfiguration sounds undesirable because it is
an
>>> >> important feature, but I currently don't use it in production, so from
>>> a
>>> >> practical point of view, I can go both ways. Whether we go through the
>>> >> trouble of doing 2 depends on users interested in that option and folks
>>> >> willing to implement it.
>>> >> >
>>> >> > To answer your question, Powell, my pseudo-proposal is kind of
a
>>> funny
>>> >> option because once the feature is stable, then we wouldn't need a
>>> switch
>>> >> any longer, so there is not need of a deprecation path, we just start
>>> >> ignoring it from the first beta release. Until it is beta, I'd say that
>>> >> default is disabled.
>>> >>
>>> >> I would argue that we need this even when it does become stable. To
me
>>> >> this is not a quality issue so much as it is an auth issue. We want
to
>>> >> make it simple for folks to run a vanilla/old ZK cluster and not worry
>>> >> about the security implications of having reconfig enabled.
>>> >>
>>> >> Patrick
>>> >>
>>> >> >
>>> >> > -Flavio
>>> >> >
>>> >> >> On 17 Mar 2016, at 17:44, powell molleti
>>> <powellm79@yahoo.com.INVALID>
>>> >> wrote:
>>> >> >>
>>> >> >> Hi Flavio,
>>> >> >>
>>> >> >> Generally do config options and command line args come under
the
>>> same
>>> >> SLA as API?. I was assuming as such hence my question. Perhaps if the
>>> >> expectation is that this config option is temporary from get go then
>>> may be
>>> >> it is ok. The default for re-config support will be enabled or
>>> disabled?.
>>> >> >>
>>> >> >> I am just thinking from provisioning point of view when people
>>> generate
>>> >> config options etc.
>>> >> >>
>>> >> >> Thanks
>>> >> >> Powell.
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <
>>> fpj@apache.org>
>>> >> wrote:
>>> >> >> Hi Powell,
>>> >> >>
>>> >> >> I was thinking config file and system property server side.
What's
>>> your
>>> >> concern with compatibility? The API itself wouldn't change, but the
>>> config
>>> >> option wouldn't exist in previous versions and would not exist either
>>> in
>>> >> later stable versions of 3.5.
>>> >> >>
>>> >> >> -Flavio
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>> On 16 Mar 2016, at 22:08, powell molleti
>>> <powellm79@yahoo.com.INVALID>
>>> >> wrote:
>>> >> >>>
>>> >> >>> Will this option be supplied via config file/args/API?.
Will this
>>> >> option be a temporary thing i.e what about its compatibility?.
>>> >> >>>
>>> >> >>> thanks
>>> >> >>> Powell.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira
<
>>> fpj@apache.org>
>>> >> wrote:
>>> >> >>> The main issue to sort out is stability of the API. There
is a
>>> >> security concern around the fact that clients can freely reconfigure
>>> the
>>> >> ensemble. If we follow the plan that Pat proposed some time ago:
>>> >> >>>
>>> >> >>>
>>> >>
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>> >> <
>>> >>
>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>>> >> >
>>> >> >>>
>>> >> >>> Locking the API is the main step to move it to beta. Sorting
out
>>> bugs
>>> >> is definitely necessary, but it isn't the main thing that is keeping
>>> 3.5 in
>>> >> alpha.
>>> >> >>>
>>> >> >>> About making it experimental, I was raising the option
of having a
>>> >> switch that disables the API calls, not the code. The reason why that
>>> could
>>> >> work is that anyone using 3.5 who uses the "experimental" API must
>>> explicit
>>> >> turn on the switch and enable the calls. If they do it, they need to
be
>>> >> aware that the API can change.
>>> >> >>>
>>> >> >>> I must say that I haven't really looked closely into doing
it, and
>>> I'm
>>> >> not even entirely convinced that this is a good idea, but since Jason
>>> >> raised the point, I'm exploring options.
>>> >> >>>
>>> >> >>> -Flavio
>>> >> >>>
>>> >> >>>
>>> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <shralex@gmail.com>
>>> wrote:
>>> >> >>>>
>>> >> >>>> Looking at the list of ~50 blocker and critical bugs
in ZooKeeper,
>>> >> only 3-4
>>> >> >>>> are related to reconfig. Given this, and the fact that
it is run
>>> in
>>> >> >>>> production since 2012 in multiple companies, I don't
think its
>>> more
>>> >> >>>> unstable than any other part of ZooKeeper.
>>> >> >>>>
>>> >> >>>> There are multiple reconfig-related bugs that turned
out really
>>> >> difficult
>>> >> >>>> to debug without access to the actual system or at
least to the
>>> Hudson
>>> >> >>>> machines where some tests are failing. In the past,
Michi and I
>>> >> physically
>>> >> >>>> went to Hortonworks to debug one such issue, but this
is of course
>>> >> not a
>>> >> >>>> good method, and we weren't able to arrange such a
visit again.
>>> >> >>>>
>>> >> >>>> Regarding making it optional - the reconfig logic has
several
>>> >> different
>>> >> >>>> parts, some would be really difficult to disable using
a
>>> configuration
>>> >> >>>> parameter. But the actual dynamic expansion of the
cluster is
>>> >> triggered by
>>> >> >>>> the reconfig command, so it should not affect users
who don't
>>> invoke
>>> >> it.
>>> >> >>>>
>>> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA
<
>>> fpj@apache.org>
>>> >> wrote:
>>> >> >>>>
>>> >> >>>>> I suppose we could give it a try. How do other
folks feel about
>>> it?
>>> >> >>>>>
>>> >> >>>>> -Flavio
>>> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jbr@squareup.com>
>>> wrote:
>>> >> >>>>>
>>> >> >>>>>> So, you could enable the dynamic reconfiguration
feature behind
>>> a
>>> >> config
>>> >> >>>>>> option, and document that it should only be
enabled
>>> experimentally,
>>> >> use
>>> >> >>>>> at
>>> >> >>>>>> your own risk.  Keep it off by default.  Allow
only static
>>> config by
>>> >> >>>>>> default, until it's stable?
>>> >> >>>>>>
>>> >> >>>>>> Jason
>>> >> >>>>>>
>>> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira
<
>>> fpj@apache.org>
>>> >> >>>>> wrote:
>>> >> >>>>>>
>>> >> >>>>>>> Hi Jason,
>>> >> >>>>>>>
>>> >> >>>>>>> The consumer in Kafka is pretty independent
from the core
>>> >> (brokers),
>>> >> >>>>>>> that's how that project manages to make
such a separation. We
>>> don't
>>> >> >>>>> have
>>> >> >>>>>>> the same with ZooKeeper as the feature
we are talking about is
>>> >> part of
>>> >> >>>>>> the
>>> >> >>>>>>> server and the only way I see of doing
what you say is to turn
>>> off
>>> >> >>>>>>> features. More specifically, we'd need
to disable the reconfig
>>> API
>>> >> and
>>> >> >>>>> do
>>> >> >>>>>>> not allow any change to the configuration,
even though the
>>> code is
>>> >> >>>>> there.
>>> >> >>>>>>>
>>> >> >>>>>>> Reconfig here refers to the ability of
changing the
>>> configuration
>>> >> of an
>>> >> >>>>>>> ensemble (e.g., changing the set of servers).
>>> >> >>>>>>>
>>> >> >>>>>>> -Flavio
>>> >> >>>>>>>
>>> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg
<jbr@squareup.com>
>>> >> wrote:
>>> >> >>>>>>>>
>>> >> >>>>>>>> So, it would seem sensible to me to
have a release where all
>>> >> features
>>> >> >>>>>> are
>>> >> >>>>>>>> stable, except where noted.  E.g. mark
certain features as
>>> only
>>> >> >>>>> 'alpha
>>> >> >>>>>>>> quality', e.g. the 're-config feature'.
 (I assume it's
>>> possible
>>> >> to
>>> >> >>>>>>> happily
>>> >> >>>>>>>> use 3.5.X without exercising the unstable
re-config bits?).
>>> >> >>>>>>>>
>>> >> >>>>>>>> There's precedent for doing this sort
of thing in other
>>> projects,
>>> >> >>>>> e.g.
>>> >> >>>>>> in
>>> >> >>>>>>>> Kafka, they've had several release
where a new "Consumer API"
>>> is
>>> >> >>>>>> shipped
>>> >> >>>>>>>> that is available for beta-testing,
but you can still just
>>> use the
>>> >> >>>>>> older
>>> >> >>>>>>>> stable consumer api, etc.
>>> >> >>>>>>>>
>>> >> >>>>>>>> Jason
>>> >> >>>>>>>>
>>> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell
molleti
>>> >> >>>>>>> <powellm79@yahoo.com.invalid
>>> >> >>>>>>>>> wrote:
>>> >> >>>>>>>>
>>> >> >>>>>>>>> Hi Doug,
>>> >> >>>>>>>>> Is 3.5 being an alpha release preventing
you from using it?.
>>> Or
>>> >> have
>>> >> >>>>>> you
>>> >> >>>>>>>>> run into issues with it?. In general
perhaps ZK 3.5 being
>>> >> labeled as
>>> >> >>>>>>> alpha
>>> >> >>>>>>>>> might not be fair, since it is
far more stable then what most
>>> >> people
>>> >> >>>>>>>>> associate an alpha release to be.
>>> >> >>>>>>>>> Perhaps if you do not use re-config
feature may be it will
>>> just
>>> >> work
>>> >> >>>>>> for
>>> >> >>>>>>>>> you?.
>>> >> >>>>>>>>> There are many examples of 3.5.X
being used in productions
>>> from
>>> >> my
>>> >> >>>>>>> limited
>>> >> >>>>>>>>> knowledge.
>>> >> >>>>>>>>> ThanksPowell.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44
AM, Flavio Junqueira <
>>> >> >>>>>>> fpj@apache.org>
>>> >> >>>>>>>>> wrote:
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> None of us expected the reconfig
changes to take this long to
>>> >> >>>>>> stabilize.
>>> >> >>>>>>>>> Until we get there, I don't think
we can do anything else
>>> with
>>> >> 3.5.
>>> >> >>>>>> The
>>> >> >>>>>>>>> best bet we have is to work harder
to bring 3.5 into a stable
>>> >> rather
>>> >> >>>>>>> than
>>> >> >>>>>>>>> trying to work around it.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> There are lots of people interested
in seeing 3.5 stable,
>>> and if
>>> >> we
>>> >> >>>>>> get
>>> >> >>>>>>>>> everyone to contribute more patches
and code reviews, we
>>> should
>>> >> be
>>> >> >>>>>> able
>>> >> >>>>>>> to
>>> >> >>>>>>>>> do it sooner. After all, it is
a community based effort, so
>>> the
>>> >> >>>>>>> community
>>> >> >>>>>>>>> shouldn't rely on just 2-3 people
doing the work.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> -Flavio
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris
Nauroth <
>>> >> cnauroth@hortonworks.com>
>>> >> >>>>>>>>> wrote:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Doug, I forgot to respond to
your question about 3.4.  Since
>>> >> 3.4 is
>>> >> >>>>>> the
>>> >> >>>>>>>>>> stable maintenance line, we
are very conservative about
>>> >> >>>>> back-porting
>>> >> >>>>>> to
>>> >> >>>>>>>>>> it.  Our policy is to limit
back-ports to critical bug
>>> fixes and
>>> >> >>>>> not
>>> >> >>>>>>>>>> introduce any new features
in the 3.4 line.  This is a
>>> matter of
>>> >> >>>>>>> managing
>>> >> >>>>>>>>>> risk.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Jason, your question about
release cadence is a fair one.
>>> If
>>> >> it's
>>> >> >>>>>> any
>>> >> >>>>>>>>>> consolation, we are now taking
the approach of trying to
>>> limit
>>> >> the
>>> >> >>>>>>> scope
>>> >> >>>>>>>>>> of anything new introduced
in 3.5 too.  That would allow us
>>> to
>>> >> >>>>> focus
>>> >> >>>>>> on
>>> >> >>>>>>>>>> stabilization: resolving blocker
bugs and freezing public
>>> >> APIs.  I
>>> >> >>>>>>> think
>>> >> >>>>>>>>>> this will help us accelerate
the releases into beta and
>>> eventual
>>> >> >>>>> GA.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> I am new to ZooKeeper release
management, so I'd like to
>>> hear
>>> >> >>>>>> thoughts
>>> >> >>>>>>>>>> from more experienced committers
and PMC members about your
>>> >> >>>>> proposal
>>> >> >>>>>> to
>>> >> >>>>>>>>>> try to cut a stable release
for a limited subset of what is
>>> in
>>> >> >>>>>>> branch-3.5
>>> >> >>>>>>>>>> now.  My instinct is that it
would be challenging to
>>> cherry-pick
>>> >> >>>>> out
>>> >> >>>>>>>>>> pieces of branch-3.5 piecemeal
at this point.  This would
>>> become
>>> >> >>>>>>> another
>>> >> >>>>>>>>>> release line for an already
resource-constrained volunteer
>>> >> staff to
>>> >> >>>>>>>>>> manage.  I'd prefer to dedicate
those limited resources to
>>> >> overall
>>> >> >>>>>> 3.5
>>> >> >>>>>>>>>> stabilization.  Also, a 3.5
release in which certain
>>> features
>>> >> >>>>>>> "vanished"
>>> >> >>>>>>>>>> because of not meeting some
stability criteria would be
>>> >> >>>>> undesirable.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> --Chris Nauroth
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason
Rosenberg" <jbr@squareup.com>
>>> >> wrote:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>> Chris,
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Can you say whether some
parts of 3.5.X are more stable
>>> than
>>> >> >>>>> others
>>> >> >>>>>>>>> (e.g.
>>> >> >>>>>>>>>>> if we don't care about
certain new features, is it
>>> relatively
>>> >> >>>>>> stable)?
>>> >> >>>>>>>>>>> Would it be possible to
cut out a version that only has the
>>> >> bits
>>> >> >>>>> we
>>> >> >>>>>>>>> think
>>> >> >>>>>>>>>>> are stable (and release
that)?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> From that timeline, and
the historic release cadence, it
>>> would
>>> >> >>>>> seem
>>> >> >>>>>> to
>>> >> >>>>>>>>> be
>>> >> >>>>>>>>>>> a
>>> >> >>>>>>>>>>> years away before we get
to the stable release?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Thanks,
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Jason
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at
1:06 PM, Chris Nauroth <
>>> >> >>>>>>>>> cnauroth@hortonworks.com>
>>> >> >>>>>>>>>>> wrote:
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>>> Hello Doug,
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> Thanks for your interest
in the SSL feature!
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> At this point, I think
we're still pretty far away from
>>> >> >>>>> declaring a
>>> >> >>>>>>>>>>>> stable
>>> >> >>>>>>>>>>>> release in the 3.5
line.  I don't think we're close enough
>>> >> that
>>> >> >>>>>>> anyone
>>> >> >>>>>>>>>>>> can
>>> >> >>>>>>>>>>>> offer a reliable ETA.
 This is an earlier thread that
>>> >> describes
>>> >> >>>>> the
>>> >> >>>>>>>>>>>> high-level strategy
for release planning in the 3.5 line:
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> https://s.apache.org/ADK1
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> The next step is a
3.5.2-alpha release.  We're working on
>>> >> >>>>>> resolving a
>>> >> >>>>>>>>>>>> few
>>> >> >>>>>>>>>>>> more blockers before
we produce a release candidate.
>>> >> Hopefully
>>> >> >>>>>> that
>>> >> >>>>>>>>>>>> will
>>> >> >>>>>>>>>>>> get done in the next
few weeks.
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> --Chris Nauroth
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM,
"Doug" <itsbehind@gmail.com> wrote:
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> I know it's only
been a few months, but I was wondering
>>> if
>>> >> there
>>> >> >>>>>>> was a
>>> >> >>>>>>>>>>>>> ballpark release
date for a stable version of 3.5.1. Or
>>> is
>>> >> there
>>> >> >>>>>> any
>>> >> >>>>>>>>>>>>> chance
>>> >> >>>>>>>>>>>>> the SSL feature
would be added to 3.4.8? Just another
>>> person
>>> >> >>>>>> looking
>>> >> >>>>>>>>> to
>>> >> >>>>>>>>>>>>> have
>>> >> >>>>>>>>>>>>> that feature in
a stable version. Thanks for all you do!
>>> :)
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> --
>>> >> >>>>>>>>>>>>> View this message
in context:
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >>
>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>>> >> >>>>>>>>>>>> e
>>> >> >>>>>>>>>>>>> -tp7581744p7582136.html
>>> >> >>>>>>>>>>>>> Sent from the zookeeper-user
mailing list archive at
>>> >> Nabble.com.
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >> >
>>> >>
>>>
>>
>>

Mime
View raw message