From dev-return-94757-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Fri Jun 1 03:00:53 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id BA411180632 for ; Fri, 1 Jun 2018 03:00:52 +0200 (CEST) Received: (qmail 90706 invoked by uid 500); 1 Jun 2018 01:00:51 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 90693 invoked by uid 99); 1 Jun 2018 01:00:50 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jun 2018 01:00:50 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 5F0C2180795 for ; Fri, 1 Jun 2018 01:00:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.979 X-Spam-Level: * X-Spam-Status: No, score=1.979 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=confluent-io.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id vOs4O5QkcsmM for ; Fri, 1 Jun 2018 01:00:46 +0000 (UTC) Received: from mail-oi0-f66.google.com (mail-oi0-f66.google.com [209.85.218.66]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 646CE5F17B for ; Fri, 1 Jun 2018 01:00:46 +0000 (UTC) Received: by mail-oi0-f66.google.com with SMTP id c128-v6so11890756oig.11 for ; Thu, 31 May 2018 18:00:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=confluent-io.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Jbh1bU8sF3dRN6yb0uwu4P1Cx7MZ2ko/G92PV2O6ils=; b=VCjcRVlJv9Vh6Nt7FpxapoZS4Vw9g3joQLz2tOTBX/uUmtWTNYf1V8Ei38y1OgVP0U 2QEo8vAunnTgzWotbzzr7EOYaav1gmd0U+9lK/W6QBEBVPTq8qSuQcE3DXyWCyaboHZC qmhghHskJKWG/Hpw1TIBb7TM2i1XfEH/CFTz671S3EdN/Wkpsn83vHAUxblO38yYYWZ8 xUKWLoDDQmuV6e4cubGeeF+O8fNua9ABKZv7XZQI0IFYAN+CdkJxjqD8rYemzrSw/Rdm DN0GXX5K/u+5RmnVjUk01gia8svA3qNBtQXSJOkrhmHBCd+Tj/amIjBQXlDhMm0AS8Cl oJrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Jbh1bU8sF3dRN6yb0uwu4P1Cx7MZ2ko/G92PV2O6ils=; b=paRML435Xdwkv5lnBS1cp47QOn17Mmb6hR6zwTgduRVbMUi8dQoj8aBAFDuJwRHqly J3M/44UQhZP0U5S4lJ6eJH2Cy8EUgEnBK1sx7V6BvmmPGqIO2FDOIq7hPFqYyNm5lOR7 J2pJ9De/ggrvtggy/Bz/M26Su7aWHs8xWABwkDiE5hEZznGE+ALrWSol7J9nClzwCXUJ RfHgmIED5UZI3OHO1aVNSJTcay6qrqTqiUwKhbjmQR8RkkSIkuEJXEY4sIZ6HsIqMVD8 YbpVsx89PUkZ08qPjDMrupg5/SMqjZeDBI9UzVT3LNYECT+LwNKrNh0OdiD7QXBJCwWL KXKg== X-Gm-Message-State: ALKqPwc8CmK1xFDvI4pF/kcRmXGB5mXuJgxTFoHP7Q3kzq4Z5M+uG4oZ fqjiRIL3DC4PLpwS19wNZ5503X9MVd2uDiUJj2UZWcsX X-Google-Smtp-Source: ADUXVKKmoU9s/+0et5TwD4uej6mseZxb1Xn8MjxtsJpf8DQKLMlgdeuEEvdxY4mEiU+eUVpbTSeLK/6Lz1hWsJ1Yiq8= X-Received: by 2002:aca:3ad6:: with SMTP id h205-v6mr4720809oia.185.1527814839821; Thu, 31 May 2018 18:00:39 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a4a:4404:0:0:0:0:0 with HTTP; Thu, 31 May 2018 18:00:39 -0700 (PDT) In-Reply-To: References: From: Jason Gustafson Date: Thu, 31 May 2018 18:00:39 -0700 Message-ID: Subject: Re: [VOTE] KIP-266: Add TimeoutException for KafkaConsumer#position To: dev Content-Type: multipart/alternative; boundary="000000000000cf855d056d8a1cfc" --000000000000cf855d056d8a1cfc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks everyone for the feedback. I've updated the KIP and added KAFKA-6979= . -Jason On Wed, May 30, 2018 at 3:50 PM, Guozhang Wang wrote: > Thanks Jason. I'm in favor of option 1 as well. > > On Wed, May 30, 2018 at 1:37 PM, Bill Bejeck wrote: > > > For what it's worth I'm +1 on Option 1 and the default value for the > > timeout. > > > > In addition to reasons outlined above by Jason, I think it will help to > > reason about consumer behavior (with respect to blocking) having the > > configuration and default value aligned with the producer. > > > > -Bill > > > > On Wed, May 30, 2018 at 3:43 PM, Ismael Juma wrote: > > > > > Sounds good to me, > > > > > > On Wed, May 30, 2018 at 12:40 PM Jason Gustafson > > > wrote: > > > > > > > Perhaps one minute? That is the default used by the producer. > > > > > > > > -Jason > > > > > > > > On Wed, May 30, 2018 at 9:50 AM, Ismael Juma > > wrote: > > > > > > > > > Option 1 sounds good to me provided that we can come up with a go= od > > > > > default. What would you suggest? > > > > > > > > > > Ismael > > > > > > > > > > On Wed, May 30, 2018 at 9:41 AM Jason Gustafson < > jason@confluent.io> > > > > > wrote: > > > > > > > > > > > Hi Everyone, > > > > > > > > > > > > There remains some inconsistency in the timeout behavior of the > > > > consumer > > > > > > APIs which do not accept a timeout. Some of them block forever > > (e.g. > > > > > > position()) and some of them use request.timeout.ms (e.g. > > > > > > parititonsFor()). > > > > > > I think we'd probably all agree that blocking forever is not > useful > > > > > > behavior and using request.timeout.ms has always been a hack > since > > > it > > > > > > controls a separate concern. I think there are basically two > > options > > > to > > > > > > address this: > > > > > > > > > > > > 1. We can add max.block.ms to match the producer and use it as > the > > > > > default > > > > > > timeout when a timeout is not explicitly provided. This will fi= x > > the > > > > > > indefinite blocking behavior and avoid conflating > > request.timeout.ms > > > . > > > > > > 2. We can deprecate the methods which don't accept a timeout. > > > > > > > > > > > > I'm leaning toward the first solution because I think we want t= o > > push > > > > > users > > > > > > to specifying timeouts through configuration rather than in cod= e > > > (Jay's > > > > > > original argument). I think the overloads are still useful for > > > advanced > > > > > > usage (e.g. in kafka streams), but we should give users an easy > > > option > > > > > with > > > > > > reasonable default behavior. > > > > > > > > > > > > If that sounds ok, I'd propose we add it to this KIP and fix it > > now. > > > > This > > > > > > gives users an easy way to get the benefit of the improvements > from > > > > this > > > > > > KIP without changing any code. > > > > > > > > > > > > Thanks, > > > > > > Jason > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, May 13, 2018 at 7:58 PM, Richard Yu < > > > > yohan.richard.yu@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > With 3 binding votes and 6 non-binding, this KIP would be > > accepted. > > > > > > > > > > > > > > Thanks for participating. > > > > > > > > > > > > > > On Thu, May 10, 2018 at 2:35 AM, Edoardo Comar < > > edocomar@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > On 10 May 2018 at 10:29, zhenya Sun wrote= : > > > > > > > > > > > > > > > > > +1 non-binding > > > > > > > > > > > > > > > > > > > =E5=9C=A8 2018=E5=B9=B45=E6=9C=8810=E6=97=A5=EF=BC=8C= =E4=B8=8B=E5=8D=885:19=EF=BC=8CManikumar > > > > =E5=86=99=E9=81=93=EF=BC=9A > > > > > > > > > > > > > > > > > > > > +1 (non-binding). > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > > > > > On Thu, May 10, 2018 at 2:33 PM, Mickael Maison < > > > > > > > > > mickael.maison@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> +1 (non binding) > > > > > > > > > >> Thanks > > > > > > > > > >> > > > > > > > > > >> On Thu, May 10, 2018 at 9:39 AM, Rajini Sivaram < > > > > > > > > > rajinisivaram@gmail.com> > > > > > > > > > >> wrote: > > > > > > > > > >>> Hi Richard, Thanks for the KIP. > > > > > > > > > >>> > > > > > > > > > >>> +1 (binding) > > > > > > > > > >>> > > > > > > > > > >>> Regards, > > > > > > > > > >>> > > > > > > > > > >>> Rajini > > > > > > > > > >>> > > > > > > > > > >>> On Wed, May 9, 2018 at 10:54 PM, Guozhang Wang < > > > > > > wangguoz@gmail.com > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > >>> > > > > > > > > > >>>> +1 from me, thanks! > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> Guozhang > > > > > > > > > >>>> > > > > > > > > > >>>> On Wed, May 9, 2018 at 10:46 AM, Jason Gustafson < > > > > > > > > jason@confluent.io> > > > > > > > > > >>>> wrote: > > > > > > > > > >>>> > > > > > > > > > >>>>> Thanks for the KIP, +1 (binding). > > > > > > > > > >>>>> > > > > > > > > > >>>>> One small correction: the KIP mentions that close() > > will > > > be > > > > > > > > > >> deprecated, > > > > > > > > > >>>> but > > > > > > > > > >>>>> we do not want to do this because it is needed by t= he > > > > > Closeable > > > > > > > > > >>>> interface. > > > > > > > > > >>>>> We only want to deprecate close(long, TimeUnit) in > > favor > > > of > > > > > > > > > >>>>> close(Duration). > > > > > > > > > >>>>> > > > > > > > > > >>>>> -Jason > > > > > > > > > >>>>> > > > > > > > > > >>>>> On Tue, May 8, 2018 at 12:43 AM, khaireddine Rezgui= < > > > > > > > > > >>>>> khaireddine120@gmail.com> wrote: > > > > > > > > > >>>>> > > > > > > > > > >>>>>> +1 > > > > > > > > > >>>>>> > > > > > > > > > >>>>>> 2018-05-07 20:35 GMT+01:00 Bill Bejeck < > > > bbejeck@gmail.com > > > > >: > > > > > > > > > >>>>>> > > > > > > > > > >>>>>>> +1 > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>> Thanks, > > > > > > > > > >>>>>>> Bill > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>> On Fri, May 4, 2018 at 7:21 PM, Richard Yu < > > > > > > > > > >>>> yohan.richard.yu@gmail.com > > > > > > > > > >>>>>> > > > > > > > > > >>>>>>> wrote: > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>>>> Hi all, I would like to bump this thread since > > > > discussion > > > > > in > > > > > > > the > > > > > > > > > >>>> KIP > > > > > > > > > >>>>>>>> appears to be reaching its conclusion. > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>> On Thu, Mar 15, 2018 at 3:30 PM, Richard Yu < > > > > > > > > > >>>>>> yohan.richard.yu@gmail.com> > > > > > > > > > >>>>>>>> wrote: > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>>>> Hi all, > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> Since there does not seem to be too much > discussion > > > in > > > > > > > > > >> KIP-266, I > > > > > > > > > >>>>>> will > > > > > > > > > >>>>>>> be > > > > > > > > > >>>>>>>>> starting a voting thread. > > > > > > > > > >>>>>>>>> Here is the link to KIP-266 for reference: > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> https://cwiki.apache.org/ > confluence/pages/viewpage > > . > > > > > > > > > >>>>>>>> action?pageId=3D75974886 > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> Recently, I have made some updates to the KIP. = To > > > > > > reiterate, > > > > > > > I > > > > > > > > > >>>> have > > > > > > > > > >>>>>>>>> included KafkaConsumer's commitSync, > > > > > > > > > >>>>>>>>> poll, and committed in the KIP. (we will be > adding > > > to a > > > > > > > > > >>>>>>> TimeoutException > > > > > > > > > >>>>>>>>> to them as well, in a similar manner > > > > > > > > > >>>>>>>>> to what we will be doing for position()) > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> Thanks, > > > > > > > > > >>>>>>>>> Richard Yu > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>>> > > > > > > > > > >>>>>>>> > > > > > > > > > >>>>>>> > > > > > > > > > >>>>>> > > > > > > > > > >>>>>> > > > > > > > > > >>>>>> > > > > > > > > > >>>>>> -- > > > > > > > > > >>>>>> Ing=C3=A9nieur en informatique > > > > > > > > > >>>>>> > > > > > > > > > >>>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> -- > > > > > > > > > >>>> -- Guozhang > > > > > > > > > >>>> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > "When the people fear their government, there is tyranny; > when > > > the > > > > > > > > government fears the people, there is liberty." [Thomas > > > Jefferson] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > -- Guozhang > --000000000000cf855d056d8a1cfc--