kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manikumar Reddy <manikumar.re...@gmail.com>
Subject Re: [VOTE] KIP-58 - Make Log Compaction Point Configurable
Date Wed, 25 May 2016 10:41:33 GMT
+1 (non binding)

On Wed, May 25, 2016 at 4:03 PM, Tom Crayford <tcrayford@heroku.com> wrote:

> +1 (non binding)
>
> Agree on log.cleaner.compaction.delay.ms being the better name.
>
> I think this setting is going to be extremely hard to tune for users, and
> worry about adding yet more configuration - Kafka already has a huge number
> of tunables though, so we're in well trod ground with "just add more
> tuning". I can't however, come up with any better mechanism (that doesn't
> require tuning at all) without gross interactions with consumer offset
> storage, so I remain a +1 here.
>
> Thanks
>
> Tom Crayford
> Heroku Kafka
>
> On Wednesday, 25 May 2016, Ewen Cheslack-Postava <ewen@confluent.io
> <javascript:_e(%7B%7D,'cvml','ewen@confluent.io');>> wrote:
>
> > +1 (binding)
> >
> > Agreed that the log.cleaner.compaction.delay.ms is probably a better
> name,
> > and consistent with log.segment.delete.delay.ms. Checked configs for
> other
> > suffixes that seemed reasonable and despite only appearing in that one
> > broker config, it seems the best match.
> >
> > -Ewen
> >
> > On Tue, May 24, 2016 at 8:16 PM, Jay Kreps <jay@confluent.io> wrote:
> >
> > > I'm +1 on the concept.
> > >
> > > As with others I think the core challenge is to express this in an
> > > intuitive way, and carry the same terminology across the docs, the
> > configs,
> > > and docstrings for the configs. Pictures would help.
> > >
> > > -Jay
> > >
> > > On Tue, May 24, 2016 at 6:54 PM, James Cheng <wushujames@gmail.com>
> > wrote:
> > >
> > > > I'm not sure what are the rules for who is allowed to vote, but I'm:
> > > >
> > > > +1 (non-binding) on the proposal
> > > >
> > > > I agree that the "log.cleaner.min.compaction.lag.ms" name is a
> little
> > > > confusing.
> > > >
> > > > I like Becket's "log.cleaner.compaction.delay.ms", or something
> > similar.
> > > >
> > > > The KIP describes it as the portion of the topic "that will remain
> > > > uncompacted", so if you're open to alternate names:
> > > >
> > > > "log.cleaner.uncompacted.range.ms"
> > > > "log.cleaner.uncompacted.head.ms" (Except that I always get "log
> tail"
> > > > and "log head" mixed up...)
> > > > "log.cleaner.uncompacted.retention.ms" (Will it be confusing to have
> > the
> > > > word "retention" in non-time-based topics?)
> > > >
> > > > I just thought of something: what happens to the value of "
> > > > log.cleaner.delete.retention.ms"? Does it still have the same
> meaning
> > as
> > > > before? Does the timer start when log compaction happens (as it
> > currently
> > > > does), so in reality, tombstones will only be removed from the log
> some
> > > > time after (log.cleaner.min.compaction.lag.ms +
> > > > log.cleaner.delete.retention.ms)?
> > > >
> > > > -James
> > > >
> > > > > On May 24, 2016, at 5:46 PM, Becket Qin <becket.qin@gmail.com>
> > wrote:
> > > > >
> > > > > +1 (non-binding) on the proposal. Just a minor suggestion.
> > > > >
> > > > > I am wondering should we change the config name to "
> > > > > log.cleaner.compaction.delay.ms"? The first glance at the
> > > configuration
> > > > > name is a little confusing. I was thinking do we have a "max" lag?
> > And
> > > is
> > > > > this "lag" a bad thing?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jiangjie (Becket) Qin
> > > > >
> > > > >
> > > > > On Tue, May 24, 2016 at 4:21 PM, Gwen Shapira <gwen@confluent.io>
> > > wrote:
> > > > >
> > > > >> +1 (binding)
> > > > >>
> > > > >> Thanks for responding to all my original concerns in the
> discussion
> > > > thread.
> > > > >>
> > > > >> On Tue, May 24, 2016 at 1:37 PM, Eric Wasserman <
> > > > eric.wasserman@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi,
> > > > >>>
> > > > >>> I would like to begin voting on KIP-58 - Make Log Compaction
> Point
> > > > >>> Configurable
> > > > >>>
> > > > >>> KIP-58 is here:  <
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-58+-+Make+Log+Compaction+Point+Configurable
> > > > >>>>
> > > > >>>
> > > > >>> The Jira ticket KAFKA-1981 Make log compaction point configurable
> > > > >>> is here: <https://issues.apache.org/jira/browse/KAFKA-1981>
> > > > >>>
> > > > >>> The original pull request is here: <
> > > > >>> https://github.com/apache/kafka/pull/1168>
> > > > >>> (this includes configurations for size and message count
lags
> that
> > > will
> > > > >> be
> > > > >>> removed per discussion of KIP-58).
> > > > >>>
> > > > >>> The vote will run for 72 hours.
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Ewen
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message