zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mocheng Guo <gmca...@gmail.com>
Subject Re: Re: complete quota system proposal
Date Tue, 23 Jul 2019 18:23:11 GMT
On Mon, Jul 22, 2019 at 12:05 AM Justin Ling Mao <maoling199210191@sina.com>
wrote:

> 1. IMHO, before that work, we need to merge the ZOOKEEPER-3301(
> https://github.com/apache/zookeeper/pull/934) firstly(:D) which has a
> good expansibility to the quota storage and support the soft/hard quota
> limit and some good refactors and utils for the quota.
>
I will help looking at the pull request.

2.In the ZOOKEEPER-1383, Thawan Kooburat has implemented a straightforward
> throughput quota based on the time windows.   A tip is:   for the soft
> limit, slow down the w/r requests; for the hard limit, just throws the
> QuotaExceededException.
>
Will include/link this PR

3.how to identify the client? cxid, hostname, ip or a unique name users
> specify? the client side supports the client group?
>
One approach is that client can send in unique id by zookeeper auth
protocol right after connection, which works without security. Another
approach is to use identity inside security certificate, which works when
tls is enabled between client/server. There can be thousands of client ips
which may change constantly and attributing traffic to ips may not be
practical.

i have created ZOOKEEPER-3467 and will continue to create subtasks as
discussed, and we can continue more specific discussion there.



>
> ----- Original Message -----
> From: Mocheng Guo <gmcatsf@gmail.com>
> To: dev@zookeeper.apache.org
> Subject: Re: complete quota system proposal
> Date: 2019-07-21 06:53
>
> Good point and I missed it. Yes client identity inside certificate can be
> used if connection is secured, otherwise we could use existing zk auth
> protocol to enable client sending its id to server.
> 0. client identification
> 1. quota configuration - metric key and value, format, storage
> 2. metrics collection and export - storage/rate/watch/connection
> 3. throttling implementation based on metrics inside server/client
> On Thu, Jul 18, 2019 at 9:18 PM Michael Han <hanm@apache.org> wrote:
> > Sounds good. I think we also need a plan on how to identify clients. If
> > auth is enabled, we might be able to reuse some of the auth information,
> > but a generalized client-id based approach sounds better.
> >
> > On Thu, Jul 18, 2019 at 11:32 AM Mocheng Guo <gmcatsf@gmail.com> wrote:
> >
> > > Hi Michael, thanks for your feedback and I will create an epic for
> this.
> > I
> > > will divide into following stories and please let me know if this makes
> > > sense
> > >
> > > 1. quota configuration - metric key and value, format, storage
> > > 2. metrics collection and export - storage/rate/watch/connection
> > > 3. throttling implementation based on metrics inside server/client
> > >
> > > On Wed, Jul 17, 2019 at 8:02 PM Michael Han <hanm@apache.org> wrote:
> > >
> > > > >> a more complete quota/throttling system would be valuable here
> > > >
> > > > Absolutely. This will improve the stability of a shared zookeeper
> > > cluster,
> > > > which is a very common use case. Because the lack of enforce quota
> (and
> > > > other soft isolation mechanisms such as flow control / request
> > limiting),
> > > > the usual practice of dealing with such excessive client usages is to
> > > > provision dedicated ZK cluster per customer which creates additional
> > > > operation overheads and is waste of resource.
> > > >
> > > > >> We'd be happy to submit the work for review
> > > >
> > > > Sounds great. I would suggest we create an epic around this and then
> > > break
> > > > down the stories to multiple sub tasks and move existing issues
> that's
> > > > relevant under the epic, so we can consolidate efforts and have a
> > single
> > > > place to track the progress.
> > > >
> > > >
> > > > On Tue, Jul 16, 2019 at 12:46 PM Mocheng Guo <gmcatsf@gmail.com>
> > wrote:
> > > >
> > > > > Hi, currently Zookeeper has a storage quota feature which is
> > > > informational
> > > > > only and there are a few JIRAs to enforce throttling.
> > > > >
> > > > > ZOOKEEPER-451
> > > > > ZOOKEEPER-2593
> > > > > ZOOKEEPER-3301
> > > > >
> > > > > I am wondering if a more complete quota/throttling system covering
> > > > metrics
> > > > > such as storage/rate/watch/connection would be valuable here? We
at
> > FB
> > > > have
> > > > > been battling with excessive system usage from clients and did some
> > > work
> > > > in
> > > > > this area. We'd be happy to submit the work for review and
> > consolidate
> > > > with
> > > > > existing efforts if people feel this is a good feature to add.
> > > > >
> > > > > thanks
> > > > > Mocheng
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message