From dev-return-107466-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Thu Sep 12 23:59:41 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 50798180651 for ; Fri, 13 Sep 2019 01:59:40 +0200 (CEST) Received: (qmail 25243 invoked by uid 500); 12 Sep 2019 23:59:38 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 25231 invoked by uid 99); 12 Sep 2019 23:59:38 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Sep 2019 23:59:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id C4B15180676 for ; Thu, 12 Sep 2019 23:59:37 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.603 X-Spam-Level: X-Spam-Status: No, score=0.603 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=confluent.io Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 54VDOU53_hoW for ; Thu, 12 Sep 2019 23:59:34 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.214.182; helo=mail-pl1-f182.google.com; envelope-from=matthias@confluent.io; receiver= Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 119F8BC50B for ; Thu, 12 Sep 2019 23:59:33 +0000 (UTC) Received: by mail-pl1-f182.google.com with SMTP id k1so12448507pls.11 for ; Thu, 12 Sep 2019 16:59:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=confluent.io; s=google; h=subject:to:references:from:openpgp:autocrypt:organization :message-id:date:user-agent:mime-version:in-reply-to; bh=/Skbf9sFBUtzZjo/VTxWQef3EYCKBUX4yv0LERNpvM8=; b=b+6V3LZ5RpNuXO3svVLPDITdRf6mavLbWuwCsbtEZmMpaHJBFXup5XeHHk/BYCo42f INkI7OaoxSEPbLiifz+qYD9LrEa+c4psMROJUCiFUomD/pxwSnoSx8+xQj0Xn/eVn/2h roZx08C/dyVbdiuSV42GABi5zOZcX+uisq0E9wZZkBc5s2fFe/LrYt3MpGY/sM5y1oQ6 2g3QXw0pVZdhLTi4pdZz8SGuL1hx8p6FRyJu3WSJe/oeF3XAPK/DX2KBeMNu8Giv0jA0 3giEXizqnlx/M7pSzbbmVCMZg6rUy9zM2qFV3l3CDHfe6y22PIrpMZ/u5jRVxBj92DFr U5xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :organization:message-id:date:user-agent:mime-version:in-reply-to; bh=/Skbf9sFBUtzZjo/VTxWQef3EYCKBUX4yv0LERNpvM8=; b=ZwK8ARB83c6s7wPo5cMOdEBLNmOmLJvltJQQpmwaAmKTcR4gzsNe/zLBTJOjzEehKb mur6cdIXlVXezCI7pRZr2yuIeD9FmsF4Xs8hY2Ljl3tRj2M6UAb/cuDT8ewAb7USUXwG 7i95FkvLocwi8SPBLxDyGoit4VHOdiCgQWcNNEKzI1nQzRSzWLxGJnf74TiVjxHMoK8y qVZs6x8ep9JJHUXI5as8qCyaE5ta06rbsBAzLPkQB6PpOcPNnWIq0GgerGmE59yUBEAX TNhzliYKPjoJ+22cO4baAF1bAaDpSBBLFkjloG2KE/3t4JCvZduGINJrKWDesyOAdQcB KXhA== X-Gm-Message-State: APjAAAXdpSzRr+A68TuuuWjLboimx6a+jRswlemDAXiS7CfL+w/mrh/X Ue4rRUj8yBXnFHvDuYFUdCW89WURrRgv60eJuA1/5UfxIiBfcB9yd04FWhebiwT4lAfWMyMy77g 6jr4h7Iwa42GRS3crYIxiDUdIXzxFAALi+VrlT/yHAZCVlIvllQjC3JT7BiQcuz8GgLA= X-Google-Smtp-Source: APXvYqwNAalX9GvaY+aIbUnjaWhB/rzgWCBzt9fPh4Bs6ioKjO0NQUsifP/Ptz7s6GWtbM6JA/207Q== X-Received: by 2002:a17:902:9689:: with SMTP id n9mr14990657plp.52.1568332766362; Thu, 12 Sep 2019 16:59:26 -0700 (PDT) Received: from Matthias-Sax-Macbook-Pro.local (50-0-2-20.static.sonic.net. [50.0.2.20]) by smtp.gmail.com with ESMTPSA id k5sm22164434pgo.45.2019.09.12.16.59.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Sep 2019 16:59:25 -0700 (PDT) Subject: Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements To: dev@kafka.apache.org References: <31f16c45-54bc-f6f3-2a94-368669decfed@confluent.io> <76823728-49ef-f06e-3459-73aa252933b9@confluent.io> From: "Matthias J. Sax" Openpgp: preference=signencrypt Autocrypt: addr=matthias@confluent.io; prefer-encrypt=mutual; keydata= mQINBFcGWisBEAD4+gj1tJcLJRckkbZjdJpd1347/Zwndn8R6r2X+YYS5EgwzP5OQHl3Q6jl hAISoqBEfDeTJffsxd1wWL+6wKU4Y7zCkH/3aL/7znOlfaewpgJP3x/naawgvnJ0jlPlJtev MlAbG+9P6aEVxYfML59KBtRKzd6OZbSh0VzCJVCvkslv+LZqR94lhA0rArupqe7EO9DuP4/V bvnDxx1dZFtEK4n4wJYsRkF+TuxGClLcfosfM0oHTZeolus+rJTi7wxrbcTOlTmOMW0Wf9rK AobXsSz838RJenQqe3X0s5EBKCoIdI2SCQiTfcJ2JTVt6Ip1IDuEVqMQmtz7i2l3Rlml0GDa gODehmeMczVIBeO0+cppzOEynjQlWLCbJ8XEjISMI87Ied6DGbEYKLnG4ucRjM//8syKI4T+ Z90Y060jhWxrvr2pGqPPaU3qvIXW1D1mchXE1ba4HOdKb6fA7j5NU47WA2YmWRDhfM4exE0Q mD3Jfjfjyuch2rGhT3twSWHk7v5zlINwOfTIfeXvShqxNzJRFf6MudFnFbgmMTo51LmPcXHz 3tUaRNoky/HRpSxU7h145SgltrKSmYgWUnG4Y3qySyiPVKfBUBi/e5dYTk4Y0NDWGhZxOXCs ZV0NQsuoqFD82LrglwECrcdHd2QaKnIX2eKB7j63dMsexFDjewARAQABtCdNYXR0aGlhcyBK LiBTYXggPG1hdHRoaWFzQGNvbmZsdWVudC5pbz6JAjgEEwECACIFAlcGWisCGwMGCwkIBwMC BhUIAgkKCwQWAgMBAh4BAheAAAoJEDwRZiHEirRPWcoP/if/UkALwOvkct/IufpQhJ9qmg/a +QrSkDbPFrkWA7r9aMnaX8C5xuFgFuekT2aMzYYEr3eQwNHeeeEEFUhJQnMsKrjMw55w8+J5 Zhzdg8OKepFT0BOovXzITZcyqBW5JdMjfwEkdztBmPHbvWCCjglSllCvNDorxF1FCYuiL+F3 aBx0SaKaXuNhA4GO2IBY68SQ04ueeSbKbnukyWdAYXObBPBvbcoJXeesX6QvANxSjCqPHRsc czAr1mADzPN58nRXrOYgeonhPROYRlhLyEJM9CnGby/GRb9WfrKFwQVVjpNT0dD9vOvobmEp o4//m6qeXh6xC/egW9vBl63fJNOb/A6T1JdFdU8VWpUofAZtPHE61Y11AmkL7EoY5eUhlL0i jCQ4+K4wERP5NGOBEHe6wfxWoQMnPwnj59N8GylCOMhaVaIYqqRuAssMWyliX8nAj0dO3Hbw EZmBMt/xAAZpXCJU3iDS1G0+fs5LIIBWJodBvjec3DpIlib3PGG6IKPfkwpTfyoStRaPAc+X 085/KhgI8hM1nCxzI/0iQSsuyrJikYcCiB+EukaxTy1TS7O3Ul7Sg6f5f2YXB3jh9rPgZf/C y8iIJDyY+zfJLaYPO3uMEtqW69SWVeLM2viy5pj3MtEzDgC16OLINRdIsibPXRchdv7KN2FZ w4uvgTnZuQINBFypIPoBEACu4ZoR57pUDJZ3UuQR/UQRetv/gZyVwhmx7zL2oA2ZLWn0GwWK ruMRdqFRh2dv0eml7W57GK2RJsvNS2hzPHDteHLgOXl5Hhg+mrP00A2srifiBN9sG1PM5tAd VwGllQcR786IiE90NP2g8C3ThCSNZCFmtVi5hmgoMoad/szOFGN7mCiSzoXETFiwmBFUfrRu Q955IMdWuS47WNBQmh631nRDrrk3hOL3NB6PPIvYHooBrTF9N0hL1qa0t4Eu06z3uiA1o0A1 cpTP7WjHyrc6wq54xTZGYX7r1c1MWQobiUg7I8W16WbZZdrKH1ZPXkHoS7w7uMyhjF5atXl+ APDtZ6fMljJ3JF/9rBgAGgSJGZj8qwDnrrA2CwXIvelztS96wJYg2WXmkH+wrTLAyZEZIpH5 vAAXeIQLV4/Fmx4RZzYVHwzuTfO4jg8Im1C22XgQ0sqbrKMzxa13ivKNBy4oRgxhxsU0iXqP wzjm1uW/t6LA61TGrwW50kNZ3rW3yG60msdRejnO5HC7kuLK9GV6cJArEJT01RRXDEmqqpG9 N7Jmit6gnCP16TPFB+QvKXab+AAwIeG6E9GcSdUAOkdD5PWv+v6XJrRkMqbfEH0j6Hc84g0z 9kCFTmnK0/zjpEhTp3zzoknGGciX1MAfxXoL/5JnukQWH3FRUPbfri3ZzwARAQABiQRyBBgB CAAmFiEEV9+AVI/wFUXaxShcPBFmIcSKtE8FAlypIPoCGwIFCQHhM4ACQAkQPBFmIcSKtE/B dCAEGQEIAB0WIQTyiy7YJwIIXl2i4ZC7w8Foa7nDUQUCXKkg+gAKCRC7w8Foa7nDUVnwD/40 R713CkSP0easwIIOyR3/oVUuTeDgtaBATYXczXLjXSYjpo6ErokhKn8cwSPhaXRGHkBMUk6b 97dQimN5p2+/PdolP8hFRvq3V9ZdcGIiK9Iq5kJlpbAV31r9Vhv7zq2wiym0LXrlwSFf+6Mn RWSIcU2Si1ySwMjXDaO7YlIj7sC/DVnlhGcPAvNgKyEt8ZZYYfWXLyrkhOzfgXInyo71XDQQ hu/8mSQwiFFyVlyw8QIWtrEPJmYHBYoBNp17qprlF/3WIiSaNDGN7Cf4Ft35XyMRLJWKsVSH FYdLJ2xxgrD58NnHXpw5ViWTHRg01CDLMjo21UaIKuLcPxsflSWkpbPULhO3Xpoiee1UxECe YBAQIRSXBdQx33SYiBdDYjBaoe/PmX3nbYf+xKEc2NOW7ClqaX/r8xh0cDiYuzOvJkvjVR6X KCi7VFcXxk7cVV65arnDFyryJ60EpNpf7BzeJxY6Lf4UfGrUNl6C//3XyKa/8eUj7RJlYROt zmSGboRNy0SJXQxThPHAMkbElEZrDe2sp4fsjlFmBDdxHojIyZnwySOrMXYPz9E4lXrVlWhs WLB9/sWWYjKWE0pq5Th6ddLLZM+blCMK6iFyQQXy83uqm9YR+eY0SGakJ1v4IVFfmO9PwqFU juMuk0Dl7IItv9pdUfYjsniPxr6Dz27TwI3NEAC5v7VvBJyilh+Honck5FbR9AUyka6D9wxb zsHWUKSQYRRXn8BfHGC+I9SLhZ+HbEwVG7069LOKz+nwOEdamZ4LDpyeRJRV9W7al2aNR66O yj+/4NG5aduk/WC8bKI6k2YkPu94abMEKHZPLXGMhbjQMlINnQaEcPBXG5FmsfPEIQs2XuMr GehVc8IQI1/o9Xn+4z69fgiXHyIEr6f+gfuSyGOaw/7gczDxxln0zUEZ2G5LUSv/RD2VaX39 PhRy7hoy9Uw4g9ZSs8OsQrjXjGTp9gX409WgDXXIss53jlCPoB59VGOsKWDBd22caG5PsdAY pLiE6JTOOoH3WHyzEshEiFbyQ1nIdFIdGzeNU+r05rtKT249SgX4fw8drym9c07PgjtSRgVa YgifOCOj9vCYgwh7Xny1B+TRX+qK3l3xrB/U6lyftZ9wdY3o2azT8uyKqG2V9nt1lFPmvJTP 2418Dglmqvpioq13zYbOUiC//k6JmSwFZR4SqAmYdsX2dh3DcIGV8CZrP1KcBsnwBgmc6Jqk zmqQ1AnWGbIyn5Bq8Ga2GtaPSUncPW+3iYAaAUWbAXODczMMFfg0UmMisyzqkbrGGymqTgVM sKLgdF+HYVyXZpbOqd/GoUDVeqAphDpypH8LABpVkIjbCnkp5UmM/gE3RU9+OP9cxzk0U7og YrkCDQRcqSFDARAAoPkZcOKC+ajTguBOSfnKykxEM8ITChN98S79pMxj6mSFi+wE3YBi/24H FCLsYcNRhDQ0LhhVDU9/e7PoGmNTFI0Eqe4yvzDVOBnUMyzhudcT3S/pDfWDCaKg/18bBQ1a c9QUJEd9PS6llv1/KUUsfP3HJBFPiJI7j7MC0cpWOf1h+hgQyRhuDn8tKEPUdQ0wdqILo9qN 5dzGf/Ilh9vcnFWdmn3TDS5nC1wFyfbZKhbm+F6paZGIIMVDUg3jI136INmSuidaFv2jJhWv BcugZ0z44s36BSkrIk4jMYN6y4xXkYWDdsyaeMdPGVkWwGkMKhLdchQGIB7LY9S6FuRZWf6Q 1N6gpJcSNX/V7ICrmUNn1zSBUXiL5oh1bhwQi4qvFS2aMzhDjcf7/Qx0LJGlGmBwB4lvyfd0 pk65Kwaqc08uNhnjlJtsdnbWfiWtRW8Y2FEwn/VGST/PiqknBYrfNh84jXpRIFVM76pduxY7 vqI8y5BZyL0o/UQkso82T+K9+V7L5k48MYPoNUdZM2xvSJ3xBgZjQP9WRQZdWxH3wf7ERaM7 d+8Z4rO/cwE+A7D3gjmfchc3o7IBJrMpwM3hITTc1gRfxHN7Fh+qQNdp6k84AAnvaFh3CSzQ MA4/VL9tzRA/ib9eBHe2r3pMp23sshA0ALS5yXIkD7J60CAjLbMAEQEAAYkCPAQYAQgAJhYh BFffgFSP8BVF2sUoXDwRZiHEirRPBQJcqSFDAhsMBQkB4TOAAAoJEDwRZiHEirRPwBkQANNw HwEEnJJiaRwrTz1u9Ie4GJG9A0tpJ5rL/DSgCdNHTmtr28oP0bjqTWnxbhBT0aGXi8tM0QJ/ isVqKfaV2Fl+32xG3a9/Y8hMqtGErZjZQjHAhBLw3tjc2/1g8EOmsb+P3AkbWNWatcejcK9U OErJvo/EzUxHLktgFSgKf6rsx4Pv910mFEczcsf8Fq75lpqV8QH+/n/gJx4enlhLOoE9MrG0 RDMbMzJMN5IDqGU3Ae/Khcya8jh5h8rSl4Y+mdXbR5yD//YXI/VK70wkk8Fe6JdA4+7DY9/s d6fi1t46D9tveO0nSnvxOdKMfoRDIyLRmZgQqLgjiC/1pJ1ZyZHMUpSvuX6Fy1G1jeAiR9Vj PVSzRMT0XtHVh6zeOPp+qIrMkuPQXeItn/t4KoY9bSCzU5YGEjmlDwQyEhVUpdi6Fa3O1vrb d4+pkVefdrAePy/W7dYiWzJKPT4lOKAFc1a9uJ5+koDb+imQd7MjX7bWFz29ocsfuqvDnKIy 08Oq3k0IhF/lGDQCekWJ6Tap3mtOrvo0m9Z3pwOhWrVdLnHAPWxXP5rJ3Pdame0pn8AFRG40 0XOX0Jn6YDBLXXynr79ZCqiSqs2VDLIP4kW3M4wYe7MsqvbGBTLyOcGnUEQMLT+mpaUJg+ac I5FRbGf3c/r2PPse2sMSKW6sZ1+MhAwK Organization: Confluent Inc Message-ID: Date: Thu, 12 Sep 2019 16:59:24 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="7aYbFUSijHbTxY5Mh6sqWA5xyJgy7fIxy" --7aYbFUSijHbTxY5Mh6sqWA5xyJgy7fIxy Content-Type: multipart/mixed; boundary="NDwde8UpwVhnmaDmTLuIirbWshIsOdjxT"; protected-headers="v1" From: "Matthias J. Sax" To: dev@kafka.apache.org Message-ID: Subject: Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements References: <31f16c45-54bc-f6f3-2a94-368669decfed@confluent.io> <76823728-49ef-f06e-3459-73aa252933b9@confluent.io> In-Reply-To: --NDwde8UpwVhnmaDmTLuIirbWshIsOdjxT Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Thanks for updating the KIP. I agree with John that we (ie, you :)) can start a vote. -Matthias On 9/11/19 9:23 AM, John Roesler wrote: > Thanks for the update, Jukka! >=20 > I'd be in favor of the current proposal. Not sure how the others feel. > If people generally feel positive, it might be time to start a vote. >=20 > Thanks, > -John >=20 > On Sat, Sep 7, 2019 at 12:40 AM Jukka Karvanen > wrote: >> >> Hi, >> >> Sorry; I need to rollback right away the one method removal what I was= >> proposing. >> >> One constructor with Long restored to TestRecord, which is needed by >> TestInputTopic. >> >> Regards, >> Jukka >> >> la 7. syysk. 2019 klo 8.06 Jukka Karvanen (jukka.karvanen@jukinimi.com= ) >> kirjoitti: >> >>> Hi, >>> >>> Let's get back to this after summer break. >>> Thanks Antony to offering help, it might be needed. >>> >>> I modified the KIP based on the feedback to be a mixture of variation= s 4 >>> and 5. >>> >>> In TestInputTopic I removed deprecation from one variation with long >>> timestamp and removed totally one version without key. >>> The existing test code with it can be easily migrated to use remainin= g >>> method adding null key. >>> >>> In TestRecord I removed constructors with Long timestamp from the pub= lic >>> interface. You can migrate existing code >>> with Long timestamp constructors to use constructors with ProducerRec= ord >>> or ConsumerRecord. >>> There is still Long timestamp(); method like in ProducerRecord / >>> ConsumerRecord. >>> >>> Is this acceptable alternative? >>> What else is needed to conclude the discussion phase and get to votin= g? >>> >>> Regards, >>> Jukka >>> >>> to 5. syysk. 2019 klo 17.17 Antony Stubbs (antony@confluent.io) kirjo= itti: >>> >>>> Hi Jukka! I just came across your work - it looks great! I was takin= g a >>>> stab at improving the existing API, but yours already looks great an= d just >>>> about complete! Are you planning on continuing your work and submitt= ing a >>>> PR? If you want some help, I'd be happy to jump in. >>>> >>>> Regards, >>>> Antony. >>>> >>>> On Thu, Aug 1, 2019 at 3:42 PM Bill Bejeck wrote= : >>>> >>>>> Hi Jukka, >>>>> >>>>> I also think 3, 4, and 5 are all good options. >>>>> >>>>> My personal preference is 4, but I also wouldn't mind going with 5 = if >>>> that >>>>> is what others want to do. >>>>> >>>>> Thanks, >>>>> Bill >>>>> >>>>> On Tue, Jul 30, 2019 at 9:31 AM John Roesler wr= ote: >>>>> >>>>>> Hey Jukka, >>>>>> >>>>>> Sorry for the delay. >>>>>> >>>>>> For what it's worth, I think 3, 4, and 5 are all good options. I >>>> guess my >>>>>> own preference is 5. >>>>>> >>>>>> It seems like the migration pain is a one-time concern vs. having = more >>>>>> maintainable code for years thereafter. >>>>>> >>>>>> Thanks, >>>>>> -John >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jul 2, 2019 at 4:03 AM Jukka Karvanen < >>>>> jukka.karvanen@jukinimi.com >>>>>>> >>>>>> wrote: >>>>>> >>>>>>> Hi Matthias, >>>>>>> >>>>>>> Generally I think using Instant and Duration make the test more >>>>> readable >>>>>>> and that's why I modified KIP based on your suggestion. >>>>>>> Now a lot of code use time with long or Long and that make the >>>> change >>>>>> more >>>>>>> complicated. >>>>>>> >>>>>>> What I tried to say about the migration is the lines without >>>> timestamp >>>>> or >>>>>>> if long timestamp is supported can be migrated mainly with search= & >>>>>>> recplace: >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> testDriver.pipeInput(recordFactory.create(WordCountLambdaExample.inp= utTopic, >>>>>>> nullKey, "Hello", 1L)); >>>>>>> >>>>>>> -> >>>>>>> >>>>>>> *inputTopic*.pipeInput(nullKey, "Hello", 1L); >>>>>>> >>>>>>> If long is not supported as timestamp, the same is not so easy: >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> testDriver.pipeInput(recordFactory.create(WordCountLambdaExample.inp= utTopic, >>>>>>> nullKey, "Hello", 1L)); >>>>>>> >>>>>>> -> >>>>>>> >>>>>>> *inputTopic1*.pipeInput(nullKey, "Hello", Instant.ofEpochMilli(1L= )); >>>>>>> >>>>>>> Also if you need to convert arbitrary long timestamps to proper t= ime >>>>>>> Instants, it require you need to understand the logic of the test= =2E >>>> So >>>>>>> mechanical search & replace is not possible. >>>>>>> >>>>>>> >>>>>>> I see there are several alternatives for long vs Instant / Durati= on: >>>>>>> >>>>>>> 1. All times as long/Long like in this version: >>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=3D1= 19550011 >>>>>>> >>>>>>> (startTimestampMs, autoAdvanceMs as parameter of createInputTopi= c >>>>>>> instead of configureTiming) >>>>>>> >>>>>>> 2. Auto timestamping configured with Instant and Duration, pipeIn= put >>>>>>> and TestRecord with long: >>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=3D1= 20722523 >>>>>>> >>>>>>> >>>>>>> 3. (CURRENT) Auto timestamping configured with Instant and Durati= on, >>>>>>> pipeInput and TestRecord with Instant, version with long deprecat= ed: >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+Topolog= yTestDriver+test+input+and+output+usability+improvements >>>>>>> >>>>>>> >>>>>>> 4. Auto timestamping configured with Instant and Duration, pipeIn= put >>>>>>> and TestRecord with Instant and long parallel (with long not >>>>>>> deprecated): >>>>>>> >>>>>>> 5. Auto timestamping configured with Instant and Duration, pipeIn= put >>>>>>> and TestRecord with Instant only >>>>>>> >>>>>>> I hope to get feedback. >>>>>>> >>>>>>> My own preference currently is alternative 3. or 4. >>>>>>> >>>>>>> >>>>>>> If somebody want to test, there is a implementation of this versi= on >>>>>>> available in Github: >>>>>>> >>>>>>> https://github.com/jukkakarvanen/kafka-streams-test-topics >>>>>>> >>>>>>> which can be used directly from public Maven repository: >>>>>>> >>>>>>> >>>>>>> com.github.jukkakarvanen >>>>>>> kafka-streams-test-topics >>>>>>> 0.0.1-beta3 >>>>>>> test >>>>>>> >>>>>>> >>>>>>> Also is this approach in KIP-470 preferred over KIP-456, so can w= e >>>>> close >>>>>>> it: >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-456%3A+Helper+= classes+to+make+it+simpler+to+write+test+logic+with+TopologyTestDriver >>>>>>> >>>>>>> Jukka >>>>>>> >>>>>>> . >>>>>>> >>>>>>> >>>>>>> pe 28. kes=C3=A4k. 2019 klo 1.10 Matthias J. Sax (matthias@conflu= ent.io) >>>>>>> kirjoitti: >>>>>>> >>>>>>>> Thanks Jukka! >>>>>>>> >>>>>>>> The idea to use `Instant/Duration` was a proposal. If we think >>>> it's >>>>> not >>>>>>>> a good one, we could still stay with `long`. Because >>>> `ProducerRecord` >>>>>>>> and `ConsumerRecord` are both based on `long,` it might make >>>> sense to >>>>>>>> keep `long`? >>>>>>>> >>>>>>>>> The result of converting millis to Instant directly generates >>>>>>>>>> rather non readable test code and changing from long to Instan= t >>>>>>>> correctly >>>>>>>>>> require understand what is the case it is testing. >>>>>>>> >>>>>>>> This might be a good indicator the using `Instant/Duration` migh= t >>>> not >>>>>> be >>>>>>>> a good idea. >>>>>>>> >>>>>>>> Would be nice to get feedback from others. >>>>>>>> >>>>>>>> About adding new methods that we deprecate immediately: I don't >>>> think >>>>>> we >>>>>>>> should do this. IMHO, there are two kind of users, one that >>>>> immediately >>>>>>>> rewrite their code to move off deprecated methods. Those won't u= se >>>>> the >>>>>>>> new+deprecated ones anyway. Other uses migrate their code slowly= >>>> and >>>>>>>> would just not rewrite their code at all, and thus also not use >>>> the >>>>>>>> new+deprecated methods. >>>>>>>> >>>>>>>>> Checking my own tests I was able to migrate the most of my code= >>>>> with >>>>>>>>> search&replace without further thinking about the logic to this= >>>> new >>>>>>>>> approach. The result of converting millis to Instant directly >>>>>> generates >>>>>>>>> rather non readable test code and changing from long to Instant= >>>>>>> correctly >>>>>>>>> require understand what is the case it is testing. >>>>>>>> >>>>>>>> Not sure if I can follow here. You first say, you could easily >>>>> migrate >>>>>>>> your code, but than you say it was not easily possible? Can you >>>>> clarify >>>>>>>> your experience upgrading your test code? >>>>>>>> >>>>>>>> >>>>>>>> -Matthias >>>>>>>> >>>>>>>> >>>>>>>> On 6/27/19 12:21 AM, Jukka Karvanen wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>>>> (4) Should we switch from `long` for timestamps to `Instant` >>>> and >>>>>>>>> `Duration` ? >>>>>>>>>> This version startTimestamp is Instant and autoAdvance >>>> Duration in >>>>>>>>> Initialization and with manual configured collection pipe >>>> methods. >>>>>>>>>> Now timestamp of TestRecord is still Long and similarly single= >>>>>> record >>>>>>>>> pipeInput still has long as parameter. >>>>>>>>>> Should these also converted to to Instant type? >>>>>>>>>> Should there be both long and Instant parallel? >>>>>>>>>> I expect there are existing test codebase which would be >>>> easier to >>>>>>>> migrate >>>>>>>>> if long could be still used. >>>>>>>>> Now added Instant version to TestRecord and pipeInput method. >>>>>>>>> >>>>>>>>> Checking my own tests I was able to migrate the most of my code= >>>>> with >>>>>>>>> search&replace without further thinking about the logic to this= >>>> new >>>>>>>>> approach. The result of converting millis to Instant directly >>>>>> generates >>>>>>>>> rather non readable test code and changing from long to Instant= >>>>>>> correctly >>>>>>>>> require understand what is the case it is testing. >>>>>>>>> >>>>>>>>> That is why version with long left still as deprecated for >>>> easier >>>>>>>> migration >>>>>>>>> for existing test. >>>>>>>>> Also TopologyTestDriver constructor and advanceWallClockTime >>>>> method >>>>>>>>> modified with same approach. >>>>>>>>> >>>>>>>>> Jukka >>>>>>>>> >>>>>>>>> >>>>>>>>> ma 24. kes=C3=A4k. 2019 klo 16.47 Bill Bejeck (bbejeck@gmail.co= m) >>>>>>> kirjoitti: >>>>>>>>> >>>>>>>>>> Hi Jukka >>>>>>>>>> >>>>>>>>>>> These topic objects are only interfacing TopologyTestDriver, >>>> not >>>>>>>>>> affecting >>>>>>>>>>> the internal functionality of it. In my plan the internal dat= a >>>>>>>> structures >>>>>>>>>>> are using those Producer/ConsumerRecords as earlier. That way= >>>> I >>>>>> don't >>>>>>>> see >>>>>>>>>>> how those could be affected. >>>>>>>>>> >>>>>>>>>> I mistakenly thought the KIP was proposing to completely remov= e >>>>>>>>>> Producer/ConsumerRecords in favor of TestRecord. But taking >>>>> another >>>>>>>> quick >>>>>>>>>> look I can see the plan for using the TestRecord objects. >>>> Thanks >>>>>> for >>>>>>>> the >>>>>>>>>> clarification. >>>>>>>>>> >>>>>>>>>> -Bill >>>>>>>>>> >>>>>>>>>> On Sat, Jun 22, 2019 at 2:26 AM Jukka Karvanen < >>>>>>>>>> jukka.karvanen@jukinimi.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> Hi Matthias, >>>>>>>>>>> >>>>>>>>>>>> (1) It's a little confusing that you list all method >>>> (existing, >>>>>>>> proposed >>>>>>>>>>>> to deprecate, and new one) of `TopologyTestDriver` in the >>>> KIP. >>>>>> Maybe >>>>>>>>>>>> only list the ones you propose to deprecate and the new ones= >>>> you >>>>>>> want >>>>>>>> to >>>>>>>>>>>> add? >>>>>>>>>>> >>>>>>>>>>> Ok. Unmodified methods removed. >>>>>>>>>>> >>>>>>>>>>>> (2) `TopologyTestDriver#createInputTopic`: might it be worth= >>>> to >>>>>> add >>>>>>>>>>>> overload to initialize the timetamp and auto-advance feature= >>>>>>> directly? >>>>>>>>>>>> Otherwise, uses always need to call `configureTiming` as an >>>>> extra >>>>>>>> call? >>>>>>>>>>> >>>>>>>>>>> Added with Instant and Duration parameters. >>>>>>>>>>> >>>>>>>>>>>> (3) `TestInputTopic#configureTiming()`: maybe rename to >>>>>>>>>>> `reconfigureTiming()` ? >>>>>>>>>>> >>>>>>>>>>> I removed this method when we have now initialization in >>>>>> constructor. >>>>>>>>>>> You can recreate TestInputTopic if needing to reconfigure >>>> timing. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> (4) Should we switch from `long` for timestamps to `Instant`= >>>> and >>>>>>>>>>> `Duration` ? >>>>>>>>>>> This version startTimestamp is Instant and autoAdvance >>>> Duration >>>>> in >>>>>>>>>>> Initialization and with manual configured collection pipe >>>>> methods. >>>>>>>>>>> Now timestamp of TestRecord is still Long and similarly singl= e >>>>>> record >>>>>>>>>>> pipeInput still has long as parameter. >>>>>>>>>>> Should these also converted to to Instant type? >>>>>>>>>>> Should there be both long and Instant parallel? >>>>>>>>>>> I expect there are existing test codebase which would be >>>> easier >>>>> to >>>>>>>>>> migrate >>>>>>>>>>> if long could be still used. >>>>>>>>>>> >>>>>>>>>>> Also should advanceWallClockTime in TopologyTestDriver >>>>> changed(or >>>>>>>> added >>>>>>>>>>> alternative) for Duration parameter. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> (5) Why do we have redundant getters? Or set with `getX()` >>>> and >>>>> one >>>>>>>>>>> set without `get`-prefix? >>>>>>>>>>> >>>>>>>>>>> The methods without get-prefix are for compatibility with >>>>>>>>>> ProducerRecord / >>>>>>>>>>> ConsumerRecord and I expect would make migration to TestRecor= d >>>>>>> easier. >>>>>>>>>>> Standard getters in TestRecord enable writing test ignoring >>>>>> selected >>>>>>>>>> fields >>>>>>>>>>> with hamcrest like this: >>>>>>>>>>> >>>>>>>>>>> assertThat(outputTopic.readRecord(), allOf( >>>>>>>>>>> hasProperty("key", equalTo(1L)), >>>>>>>>>>> hasProperty("value", equalTo("Hello")), >>>>>>>>>>> hasProperty("headers", equalTo(headers)))); >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> That's why I have currently both methods. >>>>>>>>>>> >>>>>>>>>>> Jukka >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> pe 21. kes=C3=A4k. 2019 klo 22.20 Matthias J. Sax ( >>>>>> matthias@confluent.io) >>>>>>>>>>> kirjoitti: >>>>>>>>>>> >>>>>>>>>>>> Thanks for the KIP. The idea to add InputTopic and >>>> OutputTopic >>>>>>>>>>>> abstractions is really neat! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Couple of minor comment: >>>>>>>>>>>> >>>>>>>>>>>> (1) It's a little confusing that you list all method >>>> (existing, >>>>>>>>>> proposed >>>>>>>>>>>> to deprecate, and new one) of `TopologyTestDriver` in the >>>> KIP. >>>>>> Maybe >>>>>>>>>>>> only list the ones you propose to deprecate and the new ones= >>>> you >>>>>>> want >>>>>>>>>> to >>>>>>>>>>>> add? >>>>>>>>>>>> >>>>>>>>>>>> (Or mark all existing methods clearly -- atm, I need to got >>>> back >>>>>> to >>>>>>>> the >>>>>>>>>>>> code to read the KIP and to extract what changes are >>>> proposed). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> (2) `TopologyTestDriver#createInputTopic`: might it be worth= >>>> to >>>>>> add >>>>>>>>>>>> overload to initialize the timetamp and auto-advance feature= >>>>>>> directly? >>>>>>>>>>>> Otherwise, uses always need to call `configureTiming` as an >>>>> extra >>>>>>>> call? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> (3) `TestInputTopic#configureTiming()`: maybe rename to >>>>>>>>>>>> `reconfigureTiming()` ? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> (4) Should we switch from `long` for timestamps to `Instant`= >>>> and >>>>>>>>>>>> `Duration` ? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> (5) Why do we have redundant getters? Or set with `getX()` >>>> and >>>>> one >>>>>>> set >>>>>>>>>>>> without `get`-prefix? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -Matthias >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 6/21/19 10:57 AM, Bill Bejeck wrote: >>>>>>>>>>>>> Jukka, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the KIP. I like the changes overall. >>>>>>>>>>>>> One thing I wanted to confirm, and this may be me being >>>>> paranoid, >>>>>>> but >>>>>>>>>>>> will >>>>>>>>>>>>> the changes for input/output topic affect how the >>>>>>> TopologyTestDriver >>>>>>>>>>>> works >>>>>>>>>>>>> with internal topics when there are sub-topologies created?= >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Jun 21, 2019 at 12:05 PM Guozhang Wang < >>>>>> wangguoz@gmail.com >>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> 1) Got it, could you list this class along with all its >>>>>> functions >>>>>>> in >>>>>>>>>>> the >>>>>>>>>>>>>> proposed public APIs as well? >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2) Ack, thanks! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Jun 20, 2019 at 11:27 PM Jukka Karvanen < >>>>>>>>>>>>>> jukka.karvanen@jukinimi.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Guozhang, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1) This TestRecord is new class in my proposal. So it is = a >>>>>>>>>> simplified >>>>>>>>>>>>>>> version of ProducerRecord and ConsumerRecord containing >>>> only >>>>>> the >>>>>>>>>>> fields >>>>>>>>>>>>>>> needed to test record content. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2) >>>>>>>>>>>>>>> public final TestInputTopic >>>>> createInputTopic(final >>>>>>>>>>> String >>>>>>>>>>>>>>> topicName, final Serde keySerde, final Serde >>>>> valueSerde); >>>>>>>>>>>>>>> public final TestOutputTopic >>>>>> createOutputTopic(final >>>>>>>>>>>> String >>>>>>>>>>>>>>> topicName, final Serde keySerde, final Serde >>>>> valueSerde); >>>>>>>>>>>>>>> The purpose is to create separate object for each input >>>> and >>>>>>> output >>>>>>>>>>>> topic >>>>>>>>>>>>>>> you are using. The topic name is given to >>>>>> createInput/OutputTopic >>>>>>>>>>> when >>>>>>>>>>>>>>> initialize topic object. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> For example: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> final TestInputTopic inputTopic1 =3D >>>>>>>>>>>>>>> testDriver.createInputTopic( INPUT_TOPIC, longSerde, >>>>>>> stringSerde); >>>>>>>>>>>>>>> final TestInputTopic inputTopic2 =3D >>>>>>>>>>>>>>> testDriver.createInputTopic( INPUT_TOPIC_MAP, longSerde, >>>>>>>>>>> stringSerde); >>>>>>>>>>>>>>> final TestOutputTopic outputTopic1 =3D >>>>>>>>>>>>>>> testDriver.createOutputTopic(OUTPUT_TOPIC, longSerde, >>>>>>> stringSerde); >>>>>>>>>>>>>>> final TestOutputTopic outputTopic2 =3D >>>>>>>>>>>>>>> testDriver.createOutputTopic(OUTPUT_TOPIC_MAP, >>>> stringSerde, >>>>>>>>>>>>>>> longSerde); >>>>>>>>>>>>>>> inputTopic1.pipeInput(1L, "Hello"); >>>>>>>>>>>>>>> assertThat(outputTopic1.readKeyValue(), equalTo(new >>>>>>> KeyValue<>(1L, >>>>>>>>>>>>>>> "Hello"))); >>>>>>>>>>>>>>> assertThat(outputTopic2.readKeyValue(), equalTo(new >>>>>>>>>>> KeyValue<>("Hello", >>>>>>>>>>>>>>> 1L))); >>>>>>>>>>>>>>> inputTopic2.pipeInput(1L, "Hello"); >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Jukka >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> to 20. kes=C3=A4k. 2019 klo 23.52 Guozhang Wang ( >>>>> wangguoz@gmail.com >>>>>> ) >>>>>>>>>>>>>> kirjoitti: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hello Jukka, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for writing the KIP, I have a couple of quick >>>>>> questions: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1) Is "TestRecord" an existing class that you propose to= >>>>>>>>>> piggy-back >>>>>>>>>>>> on? >>>>>>>>>>>>>>>> Right now we have a scala TestRecord case class but I >>>> doubt >>>>>> that >>>>>>>>>> was >>>>>>>>>>>>>> your >>>>>>>>>>>>>>>> proposal, or are you proposing to add a new Java class? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2) Would the new API only allow a single input / output >>>>> topic >>>>>>> with >>>>>>>>>>>>>>>> `createInput/OutputTopic`? If not, when we call pipeInpu= t >>>>> how >>>>>> to >>>>>>>>>>>>>>> determine >>>>>>>>>>>>>>>> which topic this record should be pipe to? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Guozhang >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Jun 17, 2019 at 1:34 PM John Roesler < >>>>>> john@confluent.io >>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Woah, I wasn't aware of that Hamcrest test style. >>>> Awesome! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for the updates. I look forward to hearing what >>>>> others >>>>>>>>>>> think. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Mon, Jun 17, 2019 at 4:12 AM Jukka Karvanen >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Wiki page updated: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+Topolog= yTestDriver+test+input+and+output+usability+improvements >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> ClientRecord removed and replaced with TestRecord in >>>>> method >>>>>>>>>> calls. >>>>>>>>>>>>>>>>>> TestRecordFactory removed (time tracking functionality= >>>> to >>>>> be >>>>>>>>>>>>>> included >>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> TestInputTopic) >>>>>>>>>>>>>>>>>> OutputVerifier deprecated >>>>>>>>>>>>>>>>>> TestRecord topic removed and getters added >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Getters in TestRecord enable writing test ignoring >>>>> selected >>>>>>>>>> fields >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>> hamcrest like this: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> assertThat(outputTopic.readRecord(), allOf( >>>>>>>>>>>>>>>>>> hasProperty("key", equalTo(1L)), >>>>>>>>>>>>>>>>>> hasProperty("value", equalTo("Hello")), >>>>>>>>>>>>>>>>>> hasProperty("headers", equalTo(headers)))); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Jukka >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> la 15. kes=C3=A4k. 2019 klo 1.10 John Roesler ( >>>>> john@confluent.io >>>>>> ) >>>>>>>>>>>>>>>> kirjoitti: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Sounds good. Thanks as always for considering my >>>>> feedback! >>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Fri, Jun 14, 2019 at 12:12 PM Jukka Karvanen >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Ok, I will modify KIP Public Interface in a wiki >>>> based >>>>> on >>>>>>> the >>>>>>>>>>>>>>>>> feedback. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> TestRecordFactory / ConsumerRecordFactory was used b= y >>>>>>>>>>>>>>>> TestInputTopic >>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>> the version I had with KIP456, but maybe I can merge= >>>>> That >>>>>>>>>>>>>>>>> functionality >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>> InputTopic or TestRecordFactory can kept non >>>> public >>>>>> maybe >>>>>>>>>>>>>>> moving >>>>>>>>>>>>>>>>> it to >>>>>>>>>>>>>>>>>>>> internals package. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I will make the proposal with a slim down interface.= >>>>>>>>>>>>>>>>>>>> I don't want to go to so slim as you proposed with >>>> only >>>>>>>>>>>>>>> TestRecord >>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>> List, because you then still end up doin= g >>>>>> helper >>>>>>>>>>>>>>>> methods >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>> construct List of TestRecord. >>>>>>>>>>>>>>>>>>>> The list of values is easier to write and clearer to= >>>>> read >>>>>>> than >>>>>>>>>>>>>> if >>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> need >>>>>>>>>>>>>>>>>>>> to contruct list of TestRecords. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> For example: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> final List inputValues =3D Arrays.asList( >>>>>>>>>>>>>>>>>>>> "Apache Kafka Streams Example", >>>>>>>>>>>>>>>>>>>> "Using Kafka Streams Test Utils", >>>>>>>>>>>>>>>>>>>> "Reading and Writing Kafka Topic" >>>>>>>>>>>>>>>>>>>> ); >>>>>>>>>>>>>>>>>>>> inputTopic.pipeValueList(inputValues); >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Let's check after the next iteration is it still >>>> worth >>>>>>>>>> reducing >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> methods. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Jukka >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> pe 14. kes=C3=A4k. 2019 klo 18.27 John Roesler ( >>>>>>> john@confluent.io) >>>>>>>>>>>>>>>>> kirjoitti: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, Jukka, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Ok, I buy this reasoning. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Just to echo what I think I read, you would just >>>> drop >>>>>>>>>>>>>>>> ClientRecord >>>>>>>>>>>>>>>>>>>>> from the proposal, and TestRecord would stand on it= s >>>>> own, >>>>>>>>>>>>>> with >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>> same methods and properties you proposed, and the >>>>> "input >>>>>>>>>>>>>> topic" >>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>> accept TestRecord, and the "output topic" would >>>> produce >>>>>>>>>>>>>>>> TestRecord? >>>>>>>>>>>>>>>>>>>>> Further, the "input and output topic" classes would= >>>>>>>>>>>>>> internally >>>>>>>>>>>>>>>>> handle >>>>>>>>>>>>>>>>>>>>> the conversion to and from ConsumerRecord and >>>>>>> ProducerRecord >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> pass >>>>>>>>>>>>>>>>>>>>> to and from the TopologyTestDriver? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This seems good to me. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Since the object coming out of the "output topic" i= s >>>>> much >>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>> ergonomic, I suspect we won't need the >>>> OutputVerifier >>>>> at >>>>>>> all. >>>>>>>>>>>>>>> It >>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>>>>> mostly needed because of all the extra junk in >>>>>>> ProducerRecord >>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>> need >>>>>>>>>>>>>>>>>>>>> to ignore. It seems better to just deprecate it. If= >>>> in >>>>>> the >>>>>>>>>>>>>>> future >>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>> turns out there is some actual use case for a >>>> verifier, >>>>>> we >>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>> have a >>>>>>>>>>>>>>>>>>>>> very small KIP to add one. But reading your >>>> response, I >>>>>>>>>>>>>> suspect >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>> existing test verification libraries would be >>>>> sufficient >>>>>> on >>>>>>>>>>>>>>> their >>>>>>>>>>>>>>>>> own. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Similarly, it seems like we can shrink the total >>>>>> interface >>>>>>> by >>>>>>>>>>>>>>>>> removing >>>>>>>>>>>>>>>>>>>>> the TestRecordFactory from the proposal. If >>>> TestRecord >>>>>>>>>>>>>> already >>>>>>>>>>>>>>>>> offers >>>>>>>>>>>>>>>>>>>>> all the constructors you'd want, then the only >>>> benefit >>>>> of >>>>>>> the >>>>>>>>>>>>>>>>> factory >>>>>>>>>>>>>>>>>>>>> is to auto-increment the timestamps, but then again= , >>>>> the >>>>>>>>>>>>>> "input >>>>>>>>>>>>>>>>> topic" >>>>>>>>>>>>>>>>>>>>> can already do that (e.g., it can do it if the >>>> record >>>>>>>>>>>>>> timestamp >>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>> set). >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Likewise, if the TestRecords are easy to create, >>>> then >>>>> we >>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>> need >>>>>>>>>>>>>>>>>>>>> all the redundant methods in "input topic" to pipe >>>>>> values, >>>>>>> or >>>>>>>>>>>>>>>>>>>>> key/values, or key/value/timestamp, etc. We can do >>>> with >>>>>>> just >>>>>>>>>>>>>>> two >>>>>>>>>>>>>>>>>>>>> methods, one for a single TestRecord and one for a >>>>>>> collection >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> them. >>>>>>>>>>>>>>>>>>>>> This reduces API ambiguity and also dramatically >>>>>> decreases >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> surface >>>>>>>>>>>>>>>>>>>>> area of the interface, which ultimately makes it >>>> much >>>>>>> easier >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> use. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> It's best to start with the smallest interface that= >>>>> will >>>>>> do >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> job >>>>>>>>>>>>>>>>>>>>> and expand it upon request, rather than throwing in= >>>>>>>>>>>>>> everything >>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>> think of up front. The extra stuff would be >>>> confusing >>>>> to >>>>>>>>>>>>>> people >>>>>>>>>>>>>>>>> facing >>>>>>>>>>>>>>>>>>>>> two practically identical paths to accomplish the >>>> same >>>>>>> goal, >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> it's >>>>>>>>>>>>>>>>>>>>> very difficult to slim the interface down later, >>>>> because >>>>>> we >>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>>>>>> really know which parts are more popular (i.e., no >>>> one >>>>>>>>>>>>>> submits >>>>>>>>>>>>>>>>>>>>> "feature requests" to _remove_ stuff they don't >>>> need, >>>>>> only >>>>>>> to >>>>>>>>>>>>>>>> _add_ >>>>>>>>>>>>>>>>>>>>> stuff that they need. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> But overall, I really like the structure of this >>>>> design. >>>>>>> I'm >>>>>>>>>>>>>>>> super >>>>>>>>>>>>>>>>>>>>> excited about this KIP. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Fri, Jun 14, 2019 at 2:55 AM Jukka Karvanen >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I am not a fan of swapping only ProducerRecord and= >>>>>>>>>>>>>>>>> ConsumerRecord. >>>>>>>>>>>>>>>>>>>>>> As a test writer point of view I do not want to >>>> care >>>>>> about >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> difference >>>>>>>>>>>>>>>>>>>>>> of those and >>>>>>>>>>>>>>>>>>>>>> that way I like to have object type which can be >>>> used >>>>> to >>>>>>>>>>>>>> pipe >>>>>>>>>>>>>>>>>>> records in >>>>>>>>>>>>>>>>>>>>>> and compare outputs. >>>>>>>>>>>>>>>>>>>>>> That way avoid unnecessary conversions between >>>>>>>>>>>>>> ProducerRecord >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>> ConsumerRecord. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> My initial assumption was that ProducerRecord and >>>>>>>>>>>>>>>>>>> ConsumerRecord.could >>>>>>>>>>>>>>>>>>>>>> implement the same ClientRecord >>>>>>>>>>>>>>>>>>>>>> and that way test writer could have used either of= >>>>>> those, >>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>> return type of timestamp method long vs Long is no= t >>>>>>>>>>>>>>> compatible. >>>>>>>>>>>>>>>>>>>>>> Now the main advantage of ClientRecord is no need >>>> to >>>>>>>>>>>>>>> duplicate >>>>>>>>>>>>>>>>>>>>>> OutputVerifier when it is modified from >>>> ProducerRecord >>>>>> to >>>>>>>>>>>>>>>>>>> ClientRecord. >>>>>>>>>>>>>>>>>>>>>> Generally is there need for OutputVerifier. Can we= >>>>>>>>>>>>>> deprecate >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> existing >>>>>>>>>>>>>>>>>>>>>> and use standard assertion libraries for new test.= >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> If you use hamcrest, assert-j or any other >>>> assertion >>>>>>>>>>>>>> library >>>>>>>>>>>>>>>>> for the >>>>>>>>>>>>>>>>>>>>> rest >>>>>>>>>>>>>>>>>>>>>> of the test, why not use it with these also. >>>>>>>>>>>>>>>>>>>>>> When we have these methods to access only needed >>>>> fields >>>>>> it >>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> easier >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>> write test like this: >>>>>>>>>>>>>>>>>>>>>> >>>>> assertThat(outputTopic.readValue()).isEqualTo("Hello"); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>> assertThat(outputTopic.readRecord()).isEqualTo(expectedRecord); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Only value for new OutputVerifier is when needing >>>> to >>>>>>> ignore >>>>>>>>>>>>>>>> some >>>>>>>>>>>>>>>>>>> fields >>>>>>>>>>>>>>>>>>>>>> ClientRecord actual =3D outputTopic.readRecord(); >>>>>>>>>>>>>>>>>>>>>> assertThat(actual.value()).isEqualTo("Hello"); >>>>>>>>>>>>>>>>>>>>>> assertThat(actual.key()).isEqualTo(expectedKey); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>> assertThat(actual.timestamp()).isEqualTo(expectedTimestamp); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> So if want to leave client package untouched, I >>>> would >>>>>>>>>>>>>> modify >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> methods >>>>>>>>>>>>>>>>>>>>>> with ClientRecord now in InputTopic and >>>> OutputTopic to >>>>>>> pass >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>> out >>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>> TestRecord instead. >>>>>>>>>>>>>>>>>>>>>> In that case there would be possibility to add >>>> methods >>>>>> to >>>>>>>>>>>>>>>>> TestRecord >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>> help ignore some fields in assertions like: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> assertThat(outputTopic.readRecord().getValueTimestamp()).isEqualTo(e= xpectedRecord.get >>>>>>>>>>>>>>>>>>>>>> ValueTimestamp()); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> How about this alternative? >>>>>>>>>>>>>>>>>>>>>> If this way sounds better I will modify KIP page i= n >>>>>> wiki. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Jukka >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> to 13. kes=C3=A4k. 2019 klo 18.30 John Roesler ( >>>>>>>>>>>>>> john@confluent.io >>>>>>>>>>>>>>> ) >>>>>>>>>>>>>>>>>>> kirjoitti: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hey, all, maybe we can jump-start this discussion= =2E >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I think this approach would be very ergonomic for= >>>>>>>>>>>>>> testing, >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>> help reduce boilerplate in tests. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The think I like most about it is that it mirrors= >>>> the >>>>>>>>>>>>>>> mental >>>>>>>>>>>>>>>>> model >>>>>>>>>>>>>>>>>>>>>>> that people already have from Kafka Streams, in >>>> which >>>>>> you >>>>>>>>>>>>>>>>> write to >>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>> "input topic" and then get your results from an >>>>> "output >>>>>>>>>>>>>>>>> topic". As >>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>> side benefit, the input and output topics in the >>>>>> proposal >>>>>>>>>>>>>>>> also >>>>>>>>>>>>>>>>>>> close >>>>>>>>>>>>>>>>>>>>>>> over the serdes, which makes it much less >>>> boilerplate >>>>>> for >>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>>>> code. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> If I can offer one suggestion for change, I'm not= >>>>> sure >>>>>>>>>>>>>> I'm >>>>>>>>>>>>>>>>> totally >>>>>>>>>>>>>>>>>>>>>>> sold on the need for a new abstraction >>>> "ClientRecord" >>>>>>>>>>>>>> with >>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>> implementation for tests "TestRecord". It seems >>>> like >>>>>> this >>>>>>>>>>>>>>>>>>>>>>> unnecessarily complicates the main (non-testing) >>>> data >>>>>>>>>>>>>>> model. >>>>>>>>>>>>>>>> It >>>>>>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>>>>>>>>>> like it would be sufficient, and just as >>>> ergonomic, >>>>> to >>>>>>>>>>>>>> have >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> input >>>>>>>>>>>>>>>>>>>>>>> topic accept ProducerRecords and the output topic= >>>>>> return >>>>>>>>>>>>>>>>>>>>>>> ConsumerRecords. I'm open to discussion on this >>>>> point, >>>>>>>>>>>>>>>> though. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the proposal, Jukka! >>>>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, May 20, 2019 at 7:59 AM Jukka Karvanen >>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I would like to start the discussion on KIP-470:= >>>>>>>>>>>>>>>>>>> TopologyTestDriver >>>>>>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>>>>>>>>> input and output usability improvements: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+Topolog= yTestDriver+test+input+and+output+usability+improvements >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This KIP is inspired by the Discussion in >>>> KIP-456: >>>>>>>>>>>>>> Helper >>>>>>>>>>>>>>>>>>> classes to >>>>>>>>>>>>>>>>>>>>> make >>>>>>>>>>>>>>>>>>>>>>>> it simpler to write test logic with >>>>>> TopologyTestDriver: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-456= >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> %3A+Helper+classes+to+make+it+simpler+to+write+test+logic+with+Topol= ogyTestDriver >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> The proposal in KIP-456 >>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-456%3A+Helper+= classes+to+make+it+simpler+to+write+test+logic+with+TopologyTestDriver >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>>>>>>>> to add alternate way to input and output topic, >>>> but >>>>>>>>>>>>>> this >>>>>>>>>>>>>>>> KIP >>>>>>>>>>>>>>>>>>> enhance >>>>>>>>>>>>>>>>>>>>>>> those >>>>>>>>>>>>>>>>>>>>>>>> classes and deprecate old functionality to make >>>>> clear >>>>>>>>>>>>>>>>> interface >>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>> test >>>>>>>>>>>>>>>>>>>>>>>> writer to use. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> In current KIP-470 proposal, topic objects are >>>>> created >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>> topicName and >>>>>>>>>>>>>>>>>>>>>>>> related serders. >>>>>>>>>>>>>>>>>>>>>>>> public final TestInputTopic >>>>>>>>>>>>>>>>>>> createInputTopic(final >>>>>>>>>>>>>>>>>>>>>>> String >>>>>>>>>>>>>>>>>>>>>>>> topicName, final Serde keySerde, final >>>> Serde >>>>>>>>>>>>>>>>> valueSerde); >>>>>>>>>>>>>>>>>>>>>>>> public final TestOutputTopic >>>>>>>>>>>>>>>>>>> createOutputTopic(final >>>>>>>>>>>>>>>>>>>>>>> String >>>>>>>>>>>>>>>>>>>>>>>> topicName, final Serde keySerde, final >>>> Serde >>>>>>>>>>>>>>>>> valueSerde); >>>>>>>>>>>>>>>>>>>>>>>> One thing I wondered if there way to find out >>>> topic >>>>>>>>>>>>>>> related >>>>>>>>>>>>>>>>> serde >>>>>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>>>>>>> TopologyTestDriver topology, it would simply >>>>> creation >>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> these >>>>>>>>>>>>>>>>>>> Topic >>>>>>>>>>>>>>>>>>>>>>>> objects: >>>>>>>>>>>>>>>>>>>>>>>> public final TestInputTopic >>>>>>>>>>>>>>>>>>> createInputTopic(final >>>>>>>>>>>>>>>>>>>>>>> String >>>>>>>>>>>>>>>>>>>>>>>> topicName); >>>>>>>>>>>>>>>>>>>>>>>> public final TestOutputTopic >>>>>>>>>>>>>>>>>>> createOutputTopic(final >>>>>>>>>>>>>>>>>>>>>>> String >>>>>>>>>>>>>>>>>>>>>>>> topicName); >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> KIP-456 can be discarded if this KIP get >>>> accepted. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>>>>>>>>> Jukka >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> -- Guozhang >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> -- Guozhang >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Antony Stubbs >>>> >>>> Solutions Architect | Confluent >>>> >>>> +44 (0)7491 833 814 <+447491833814> >>>> Follow us: Twitter | blog >>>> >>>> >>> >=20 --NDwde8UpwVhnmaDmTLuIirbWshIsOdjxT-- --7aYbFUSijHbTxY5Mh6sqWA5xyJgy7fIxy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIzBAEBCgAdFiEE8osu2CcCCF5douGQu8PBaGu5w1EFAl1629wACgkQu8PBaGu5 w1EmphAAi3CU3kKQ6cf3WI5qdKhgHfBIyGQcj8dlwLf3gz6yvN5zSTo6SYyqo40U S0LdL2BQ4cPa7lkwU2WFpQCOufRuSBsSXsKGxuYHxQKux7bpnxqDqz4jR8Vt4ROz NL6PHpVjYgf9l/xFyNJ88O3I+zX384VXwP0F5pekemnYSMCBuE7BgIToXfylGikG zF/S0O8/NYjRt/fm+7Zv1t7/pbvugxo7SKSF+Bb4dZFThe7+2//JfQUiSYS7R/9C ez3ehhccBbkiFdbkomcYw1m+qg1jQ0nSTaJD2zVGUlrwMiy5dOsgY6GO+E0oQCzn dKn8hDsI+oqx2FPD19o9ZnGtJIJpmwHhGNZh3TPfavmHQRjftHdWcQPf4JaDDm9I FftpwgBSA130il4blYFzalNsoE5CV3Uq1YGMqxUqUaCitCRx3bYCahkQ2nmI/Z9M Pv/5Qrp9SYHHWEQ85QreK/24TcqBOFfXnh3x23E7GXAq7DWBbp/OBuH6SHTM0IMG 2Y/qYus57ljS7VuzCxTfnl0YlhyhZa2T86f64Dqvn9OBG8b4xKJT3bhMorjfIRvx /C9rItrSOHmTK6k0mRHpJHP/pb5T+FzEpT/lNuv3pNLV/Z1ekfMB2jt3TO2xxLNV 1IiJBVL/vqvkYJ/cdSN97f+YpPHX+y2BiCmtsXueHE6oR7zexhM= =HVbE -----END PGP SIGNATURE----- --7aYbFUSijHbTxY5Mh6sqWA5xyJgy7fIxy--