zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Zookeeper with SSL release date
Date Tue, 22 Mar 2016 04:04:48 GMT
On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <shralex@gmail.com> wrote:
> Hi Patrick, Flavio,
>
> Since there seems to be consensus on this, I can add this flag, unless
> someone else wants to. I assume that getConfig should still work regardless
> of the flag ? is there a security concern with clients knowing the list of
> servers?
>

We've always hidden that detail from users. We don't even expose which
server you're connected to today. I remember Ben (and perhaps Flavio?)
highlighting this as important to maintain although I'm not super
familiar with the specifics on why. It made sense to me though from
the perspective that we don't want clients to make assumptions that
probably shouldn't.

My thinking is that we should 1) add a config option to enable
reconfig (off by default), 2) move reconfig specific functionality
from ZooKeeper.java (including getconfig) into an "admin" package,
within say a class ZooKeeperAdmin, 3) document/test use of ACLs for
when folks do want to enable reconfig and are also worried about auth.
(e.g. turn on kerb)

Again, I don't see any of this as a quality issue personally. As such
I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we
were interested in doing such a release. Adjusting the API should be
done before we move to "beta" though. Although that seems like a
pretty mechanical (eclipse/idea) type refactoring?

Patrick

> Cheers,
> Alex
> On Mar 21, 2016 8:34 PM, "Patrick Hunt" <phunt@apache.org> wrote:
>
>> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <fpj@apache.org> wrote:
>> > I gotta say that I'm not super excited about this option, but for some
>> reason I ended up carrying the flag. To recap, I just raised this option
>> because it seems that there are folks interested in features in 3.5 like
>> SSL and not necessarily in reconfiguration. SSL is important and to take
>> Kafka as an example, it sucks that we can't have a whole set up using SSL.
>> For ZK, the real questions are:
>> >
>> > 1- how fast can we make 3.5 stable?
>> > 2- would it be faster if we have a way of disabling reconfiguration?
>> > 3- would enough users care about a stable 3.5 that has reconfiguration
>> disabled?
>> >
>> > It is taking a long time to get 3.5 to beta. There has been some good
>> activity around 3.5.2 release, which is a great step, but it is unclear
>> when 3.5.3 is going to come and if we will be able to make 3.5 beta then.
>> Frankly, disabling reconfiguration sounds undesirable because it is an
>> important feature, but I currently don't use it in production, so from a
>> practical point of view, I can go both ways. Whether we go through the
>> trouble of doing 2 depends on users interested in that option and folks
>> willing to implement it.
>> >
>> > To answer your question, Powell, my pseudo-proposal is kind of a funny
>> option because once the feature is stable, then we wouldn't need a switch
>> any longer, so there is not need of a deprecation path, we just start
>> ignoring it from the first beta release. Until it is beta, I'd say that
>> default is disabled.
>>
>> I would argue that we need this even when it does become stable. To me
>> this is not a quality issue so much as it is an auth issue. We want to
>> make it simple for folks to run a vanilla/old ZK cluster and not worry
>> about the security implications of having reconfig enabled.
>>
>> Patrick
>>
>> >
>> > -Flavio
>> >
>> >> On 17 Mar 2016, at 17:44, powell molleti <powellm79@yahoo.com.INVALID>
>> wrote:
>> >>
>> >> Hi Flavio,
>> >>
>> >> Generally do config options and command line args come under the same
>> SLA as API?. I was assuming as such hence my question. Perhaps if the
>> expectation is that this config option is temporary from get go then may be
>> it is ok. The default for re-config support will be enabled or disabled?.
>> >>
>> >> I am just thinking from provisioning point of view when people generate
>> config options etc.
>> >>
>> >> Thanks
>> >> Powell.
>> >>
>> >>
>> >>
>> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <fpj@apache.org>
>> wrote:
>> >> Hi Powell,
>> >>
>> >> I was thinking config file and system property server side. What's your
>> concern with compatibility? The API itself wouldn't change, but the config
>> option wouldn't exist in previous versions and would not exist either in
>> later stable versions of 3.5.
>> >>
>> >> -Flavio
>> >>
>> >>
>> >>
>> >>> On 16 Mar 2016, at 22:08, powell molleti <powellm79@yahoo.com.INVALID>
>> wrote:
>> >>>
>> >>> Will this option be supplied via config file/args/API?. Will this
>> option be a temporary thing i.e what about its compatibility?.
>> >>>
>> >>> thanks
>> >>> Powell.
>> >>>
>> >>>
>> >>>
>> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <fpj@apache.org>
>> wrote:
>> >>> The main issue to sort out is stability of the API. There is a
>> security concern around the fact that clients can freely reconfigure the
>> ensemble. If we follow the plan that Pat proposed some time ago:
>> >>>
>> >>>
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> <
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzNioGK70pg+iHMHPigYfJDSLf9-eq6Q@mail.gmail.com%3E
>> >
>> >>>
>> >>> Locking the API is the main step to move it to beta. Sorting out bugs
>> is definitely necessary, but it isn't the main thing that is keeping 3.5 in
>> alpha.
>> >>>
>> >>> About making it experimental, I was raising the option of having a
>> switch that disables the API calls, not the code. The reason why that could
>> work is that anyone using 3.5 who uses the "experimental" API must explicit
>> turn on the switch and enable the calls. If they do it, they need to be
>> aware that the API can change.
>> >>>
>> >>> I must say that I haven't really looked closely into doing it, and I'm
>> not even entirely convinced that this is a good idea, but since Jason
>> raised the point, I'm exploring options.
>> >>>
>> >>> -Flavio
>> >>>
>> >>>
>> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <shralex@gmail.com>
wrote:
>> >>>>
>> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper,
>> only 3-4
>> >>>> are related to reconfig. Given this, and the fact that it is run
in
>> >>>> production since 2012 in multiple companies, I don't think its more
>> >>>> unstable than any other part of ZooKeeper.
>> >>>>
>> >>>> There are multiple reconfig-related bugs that turned out really
>> difficult
>> >>>> to debug without access to the actual system or at least to the
Hudson
>> >>>> machines where some tests are failing. In the past, Michi and I
>> physically
>> >>>> went to Hortonworks to debug one such issue, but this is of course
>> not a
>> >>>> good method, and we weren't able to arrange such a visit again.
>> >>>>
>> >>>> Regarding making it optional - the reconfig logic has several
>> different
>> >>>> parts, some would be really difficult to disable using a configuration
>> >>>> parameter. But the actual dynamic expansion of the cluster is
>> triggered by
>> >>>> the reconfig command, so it should not affect users who don't invoke
>> it.
>> >>>>
>> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <fpj@apache.org>
>> wrote:
>> >>>>
>> >>>>> I suppose we could give it a try. How do other folks feel about
it?
>> >>>>>
>> >>>>> -Flavio
>> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <jbr@squareup.com>
wrote:
>> >>>>>
>> >>>>>> So, you could enable the dynamic reconfiguration feature
behind a
>> config
>> >>>>>> option, and document that it should only be enabled experimentally,
>> use
>> >>>>> at
>> >>>>>> your own risk.  Keep it off by default.  Allow only static
config by
>> >>>>>> default, until it's stable?
>> >>>>>>
>> >>>>>> Jason
>> >>>>>>
>> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <fpj@apache.org>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Hi Jason,
>> >>>>>>>
>> >>>>>>> The consumer in Kafka is pretty independent from the
core
>> (brokers),
>> >>>>>>> that's how that project manages to make such a separation.
We don't
>> >>>>> have
>> >>>>>>> the same with ZooKeeper as the feature we are talking
about is
>> part of
>> >>>>>> the
>> >>>>>>> server and the only way I see of doing what you say
is to turn off
>> >>>>>>> features. More specifically, we'd need to disable the
reconfig API
>> and
>> >>>>> do
>> >>>>>>> not allow any change to the configuration, even though
the code is
>> >>>>> there.
>> >>>>>>>
>> >>>>>>> Reconfig here refers to the ability of changing the
configuration
>> of an
>> >>>>>>> ensemble (e.g., changing the set of servers).
>> >>>>>>>
>> >>>>>>> -Flavio
>> >>>>>>>
>> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <jbr@squareup.com>
>> wrote:
>> >>>>>>>>
>> >>>>>>>> So, it would seem sensible to me to have a release
where all
>> features
>> >>>>>> are
>> >>>>>>>> stable, except where noted.  E.g. mark certain features
as only
>> >>>>> 'alpha
>> >>>>>>>> quality', e.g. the 're-config feature'.  (I assume
it's possible
>> to
>> >>>>>>> happily
>> >>>>>>>> use 3.5.X without exercising the unstable re-config
bits?).
>> >>>>>>>>
>> >>>>>>>> There's precedent for doing this sort of thing in
other projects,
>> >>>>> e.g.
>> >>>>>> in
>> >>>>>>>> Kafka, they've had several release where a new "Consumer
API" is
>> >>>>>> shipped
>> >>>>>>>> that is available for beta-testing, but you can
still just use the
>> >>>>>> older
>> >>>>>>>> stable consumer api, etc.
>> >>>>>>>>
>> >>>>>>>> Jason
>> >>>>>>>>
>> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti
>> >>>>>>> <powellm79@yahoo.com.invalid
>> >>>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi Doug,
>> >>>>>>>>> Is 3.5 being an alpha release preventing you
from using it?. Or
>> have
>> >>>>>> you
>> >>>>>>>>> run into issues with it?. In general perhaps
ZK 3.5 being
>> labeled as
>> >>>>>>> alpha
>> >>>>>>>>> might not be fair, since it is far more stable
then what most
>> people
>> >>>>>>>>> associate an alpha release to be.
>> >>>>>>>>> Perhaps if you do not use re-config feature
may be it will just
>> work
>> >>>>>> for
>> >>>>>>>>> you?.
>> >>>>>>>>> There are many examples of 3.5.X being used
in productions from
>> my
>> >>>>>>> limited
>> >>>>>>>>> knowledge.
>> >>>>>>>>> ThanksPowell.
>> >>>>>>>>>
>> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio
Junqueira <
>> >>>>>>> fpj@apache.org>
>> >>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> None of us expected the reconfig changes to
take this long to
>> >>>>>> stabilize.
>> >>>>>>>>> Until we get there, I don't think we can do
anything else with
>> 3.5.
>> >>>>>> The
>> >>>>>>>>> best bet we have is to work harder to bring
3.5 into a stable
>> rather
>> >>>>>>> than
>> >>>>>>>>> trying to work around it.
>> >>>>>>>>>
>> >>>>>>>>> There are lots of people interested in seeing
3.5 stable, and if
>> we
>> >>>>>> get
>> >>>>>>>>> everyone to contribute more patches and code
reviews, we should
>> be
>> >>>>>> able
>> >>>>>>> to
>> >>>>>>>>> do it sooner. After all, it is a community based
effort, so the
>> >>>>>>> community
>> >>>>>>>>> shouldn't rely on just 2-3 people doing the
work.
>> >>>>>>>>>
>> >>>>>>>>> -Flavio
>> >>>>>>>>>
>> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth
<
>> cnauroth@hortonworks.com>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Doug, I forgot to respond to your question
about 3.4.  Since
>> 3.4 is
>> >>>>>> the
>> >>>>>>>>>> stable maintenance line, we are very conservative
about
>> >>>>> back-porting
>> >>>>>> to
>> >>>>>>>>>> it.  Our policy is to limit back-ports to
critical bug fixes and
>> >>>>> not
>> >>>>>>>>>> introduce any new features in the 3.4 line.
 This is a matter of
>> >>>>>>> managing
>> >>>>>>>>>> risk.
>> >>>>>>>>>>
>> >>>>>>>>>> Jason, your question about release cadence
is a fair one.  If
>> it's
>> >>>>>> any
>> >>>>>>>>>> consolation, we are now taking the approach
of trying to limit
>> the
>> >>>>>>> scope
>> >>>>>>>>>> of anything new introduced in 3.5 too. 
That would allow us to
>> >>>>> focus
>> >>>>>> on
>> >>>>>>>>>> stabilization: resolving blocker bugs and
freezing public
>> APIs.  I
>> >>>>>>> think
>> >>>>>>>>>> this will help us accelerate the releases
into beta and eventual
>> >>>>> GA.
>> >>>>>>>>>>
>> >>>>>>>>>> I am new to ZooKeeper release management,
so I'd like to hear
>> >>>>>> thoughts
>> >>>>>>>>>> from more experienced committers and PMC
members about your
>> >>>>> proposal
>> >>>>>> to
>> >>>>>>>>>> try to cut a stable release for a limited
subset of what is in
>> >>>>>>> branch-3.5
>> >>>>>>>>>> now.  My instinct is that it would be challenging
to cherry-pick
>> >>>>> out
>> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point.
 This would become
>> >>>>>>> another
>> >>>>>>>>>> release line for an already resource-constrained
volunteer
>> staff to
>> >>>>>>>>>> manage.  I'd prefer to dedicate those limited
resources to
>> overall
>> >>>>>> 3.5
>> >>>>>>>>>> stabilization.  Also, a 3.5 release in which
certain features
>> >>>>>>> "vanished"
>> >>>>>>>>>> because of not meeting some stability criteria
would be
>> >>>>> undesirable.
>> >>>>>>>>>>
>> >>>>>>>>>> --Chris Nauroth
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg"
<jbr@squareup.com>
>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Chris,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Can you say whether some parts of 3.5.X
are more stable than
>> >>>>> others
>> >>>>>>>>> (e.g.
>> >>>>>>>>>>> if we don't care about certain new features,
is it relatively
>> >>>>>> stable)?
>> >>>>>>>>>>> Would it be possible to cut out a version
that only has the
>> bits
>> >>>>> we
>> >>>>>>>>> think
>> >>>>>>>>>>> are stable (and release that)?
>> >>>>>>>>>>>
>> >>>>>>>>>>> From that timeline, and the historic
release cadence, it would
>> >>>>> seem
>> >>>>>> to
>> >>>>>>>>> be
>> >>>>>>>>>>> a
>> >>>>>>>>>>> years away before we get to the stable
release?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Jason
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris
Nauroth <
>> >>>>>>>>> cnauroth@hortonworks.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hello Doug,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks for your interest in the
SSL feature!
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> At this point, I think we're still
pretty far away from
>> >>>>> declaring a
>> >>>>>>>>>>>> stable
>> >>>>>>>>>>>> release in the 3.5 line.  I don't
think we're close enough
>> that
>> >>>>>>> anyone
>> >>>>>>>>>>>> can
>> >>>>>>>>>>>> offer a reliable ETA.  This is an
earlier thread that
>> describes
>> >>>>> the
>> >>>>>>>>>>>> high-level strategy for release
planning in the 3.5 line:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> https://s.apache.org/ADK1
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> The next step is a 3.5.2-alpha release.
 We're working on
>> >>>>>> resolving a
>> >>>>>>>>>>>> few
>> >>>>>>>>>>>> more blockers before we produce
a release candidate.
>> Hopefully
>> >>>>>> that
>> >>>>>>>>>>>> will
>> >>>>>>>>>>>> get done in the next few weeks.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --Chris Nauroth
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <itsbehind@gmail.com>
wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> I know it's only been a few
months, but I was wondering if
>> there
>> >>>>>>> was a
>> >>>>>>>>>>>>> ballpark release date for a
stable version of 3.5.1. Or is
>> there
>> >>>>>> any
>> >>>>>>>>>>>>> chance
>> >>>>>>>>>>>>> the SSL feature would be added
to 3.4.8? Just another person
>> >>>>>> looking
>> >>>>>>>>> to
>> >>>>>>>>>>>>> have
>> >>>>>>>>>>>>> that feature in a stable version.
Thanks for all you do! :)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> View this message in context:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat
>> >>>>>>>>>>>> e
>> >>>>>>>>>>>>> -tp7581744p7582136.html
>> >>>>>>>>>>>>> Sent from the zookeeper-user
mailing list archive at
>> Nabble.com.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >
>>

Mime
View raw message