zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mocheng Guo <gmca...@gmail.com>
Subject Re: complete quota system proposal
Date Fri, 19 Jul 2019 18:26:19 GMT
Good point and I missed it. Yes client identity inside certificate can be
used if connection is secured, otherwise we could use existing zk auth
protocol to enable client sending its id to server.

0. client identification
1. quota configuration - metric key and value, format, storage
2. metrics collection and export - storage/rate/watch/connection
3. throttling implementation based on metrics inside server/client


On Thu, Jul 18, 2019 at 9:18 PM Michael Han <hanm@apache.org> wrote:

> Sounds good. I think we also need a plan on how to identify clients. If
> auth is enabled, we might be able to reuse some of the auth information,
> but a generalized client-id based approach sounds better.
>
> On Thu, Jul 18, 2019 at 11:32 AM Mocheng Guo <gmcatsf@gmail.com> wrote:
>
> > Hi Michael, thanks for your feedback and I will create an epic for this.
> I
> > will divide into following stories and please let me know if this makes
> > sense
> >
> > 1. quota configuration - metric key and value, format, storage
> > 2. metrics collection and export - storage/rate/watch/connection
> > 3. throttling implementation based on metrics inside server/client
> >
> > On Wed, Jul 17, 2019 at 8:02 PM Michael Han <hanm@apache.org> wrote:
> >
> > > >> a more complete quota/throttling system would be valuable here
> > >
> > > Absolutely. This will improve the stability of a shared zookeeper
> > cluster,
> > > which is a very common use case. Because the lack of enforce quota (and
> > > other soft isolation mechanisms such as flow control / request
> limiting),
> > > the usual practice of dealing with such excessive client usages is to
> > > provision dedicated ZK cluster per customer which creates additional
> > > operation overheads and is waste of resource.
> > >
> > > >> We'd be happy to submit the work for review
> > >
> > > Sounds great. I would suggest we create an epic around this and then
> > break
> > > down the stories to multiple sub tasks and move existing issues that's
> > > relevant under the epic, so we can consolidate efforts and have a
> single
> > > place to track the progress.
> > >
> > >
> > > On Tue, Jul 16, 2019 at 12:46 PM Mocheng Guo <gmcatsf@gmail.com>
> wrote:
> > >
> > > > Hi, currently Zookeeper has a storage quota feature which is
> > > informational
> > > > only and there are a few JIRAs to enforce throttling.
> > > >
> > > > ZOOKEEPER-451
> > > > ZOOKEEPER-2593
> > > > ZOOKEEPER-3301
> > > >
> > > > I am wondering if a more complete quota/throttling system covering
> > > metrics
> > > > such as storage/rate/watch/connection would be valuable here? We at
> FB
> > > have
> > > > been battling with excessive system usage from clients and did some
> > work
> > > in
> > > > this area. We'd be happy to submit the work for review and
> consolidate
> > > with
> > > > existing efforts if people feel this is a good feature to add.
> > > >
> > > > thanks
> > > > Mocheng
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message