kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevo Slavić <ssla...@gmail.com>
Subject Re: [DISCUSS] KIP-186: Increase offsets retention default to 7 days
Date Wed, 16 Aug 2017 12:09:58 GMT
+1 for making consistent default log and offsets retention time.
I like Stephane's suggestion too, log retention override should override
offset retention too if not explicitly configured.

Please consider additionally:
- introducing offsets.retention.hours config property
- syncing log and offsets retention.check.interval.ms, if there's no real
reason for the two to differ
-- consider making retention check interval by default (if not explicitly
configured) a fraction of retention time
- name all "offsets" configs with "offsets" prefix (now it's a mix of
singular/"offset" and plural/"offsets")


On Fri, Aug 11, 2017 at 2:01 AM, Guozhang Wang <wangguoz@gmail.com> wrote:

> +1 from me
>
> On Wed, Aug 9, 2017 at 9:40 AM, Jason Gustafson <jason@confluent.io>
> wrote:
>
> > +1 on the bump to 7 days. Wanted to mention one minor point. The
> > OffsetCommit RPC still provides the ability to set the retention time
> from
> > the client, but we do not use it in the consumer. Should we consider
> adding
> > a consumer config to set this? Given the problems people had with the old
> > default, such a config would probably have gotten a fair bit of use.
> Maybe
> > it's less necessary with the new default, but there may be situations
> where
> > you don't want to keep the offsets for too long. For example, the console
> > consumer commits offsets with a generated group id. We might want to set
> a
> > low retention time to keep it from filling the offset cache with garbage
> > from such groups.
> >
>
> I agree with Jason here, but maybe itself deserves a separate KIP
> discussion.
>
>
> >
> > -Jason
> >
> > On Wed, Aug 9, 2017 at 5:24 AM, Sönke Liebau <
> > soenke.liebau@opencore.com.invalid> wrote:
> >
> > > Just had this create issues at a customer as well, +1
> > >
> > > On Wed, Aug 9, 2017 at 11:46 AM, Mickael Maison <
> > mickael.maison@gmail.com>
> > > wrote:
> > >
> > > > Yes the current default is too short, +1
> > > >
> > > > On Wed, Aug 9, 2017 at 8:56 AM, Ismael Juma <ismael@juma.me.uk>
> wrote:
> > > > > Thanks for the KIP, +1 from me.
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Wed, Aug 9, 2017 at 1:24 AM, Ewen Cheslack-Postava <
> > > ewen@confluent.io
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> I posted a simple new KIP for a problem we see with a lot of
> users:
> > > > >> KIP-186: Increase offsets retention default to 7 days
> > > > >>
> > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > >> 186%3A+Increase+offsets+retention+default+to+7+days
> > > > >>
> > > > >> Note that in addition to the KIP text itself, the linked JIRA
> > already
> > > > >> existed and has a bunch of discussion on the subject.
> > > > >>
> > > > >> -Ewen
> > > > >>
> > > >
> > >
> > >
> > >
> > > --
> > > Sönke Liebau
> > > Partner
> > > Tel. +49 179 7940878
> > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
> > >
> >
>
>
>
> --
> -- Guozhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message