From dev-return-8143-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Sat Apr 13 09:59:14 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id A2A4F180668 for ; Sat, 13 Apr 2019 11:59:14 +0200 (CEST) Received: (qmail 34464 invoked by uid 500); 13 Apr 2019 09:59:13 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 34452 invoked by uid 99); 13 Apr 2019 09:59:13 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Apr 2019 09:59:13 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 1ADA2C2319 for ; Sat, 13 Apr 2019 09:59:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id eGoNxtSCYF6n for ; Sat, 13 Apr 2019 09:59:10 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 1CBCF5F1CF for ; Sat, 13 Apr 2019 09:49:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1555148962; bh=k4ix8y01aeNWZ7FPdrEcHMbS1fLWrmGO119DIOovqmo=; h=X-UI-Sender-Class:Subject:To:References:From:Date:In-Reply-To; b=a7uhIkqyLHcbzmdA3ovUmV3uYSAzcMBJLw/gTiCCBNI3I1oTTQhQ5GdtRAYCZwHcW IN1tV7fGLZk1bBWwkh3lPfL9kSLU4GgSct2iMtUnw3WoAHGIR5b8XKekoK6+R72apT cKDLW6r+MfUzyPoeroEX7MSMe1Xc/R6kpnbcnfCg= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.0.9] ([95.90.238.44]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LyR1G-1glz1Q07io-015utR for ; Sat, 13 Apr 2019 11:49:22 +0200 Subject: Re: [2.0 spring cleaning] Deprecate contrib folder? To: dev@airflow.apache.org References: <24F53EDA-5860-4B6B-A305-8DA09672A1FA@gmail.com> <6722A60B-8225-4A65-9786-FCA3A0C515D5@gmail.com> From: Felix Uellendall Message-ID: <4fa41ee6-f6b6-acfa-80fb-16d5fce13e27@gmx.de> Date: Sat, 13 Apr 2019 11:49:21 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:aMt7mAisLDYTlXASLhbWRaVsDGMyE6NwloAAiUdckBow62az3i/ 716t4DO38u76vjupHBHU94gxdgkU0Bq160Tbs0Fp+i3NO02NMut5m9Zb+NptBiQOe1BDlxb 7NQN1kprY1QDnBpMuxbtUTS31o/EV6wEzGaDV1wbvhYhyVzAVj7cROmXOjZyzE5lYKjbM3L csjE33dkiulsLj46LDUbg== X-UI-Out-Filterresults: notjunk:1;V03:K0:QQws+sRm9SQ=:gcspt3FohU1+1AWYMvcJrj 5Z1s38Q1sdIiH71RHmKoRHjCo8MawtmkgdLw0Qy9zrjQtuIp7U92kgqTRwfShrgSGoIUxUPlh 1iaIAkdGV/tpXQtZSVU/jVv7TogY4GTYW6igl+54A09+ognmz1YKRYTQWpkr/82p/geOYY38f THRWwAVBQSQIy/daxHhaoJEtzYZKxow/Yu34Gz12KO7KCSmkK067lBRkx0Uu1Lqwc+3ZjRYfZ ViY/GYieBajR4buv5lleV8LPaSuY8oWHi5FMMkU13idj4g/2MhDJlxbJybVs48wM5MaNr2Hwf jNGrYwgdQkLj/7XJZP/nDhV3tJRezA8df24c5zO2D02gdxqlDNwazbvCgpJ46L1PaMGAHLi+m t9EqjsXMhrQ8twj1B5eaNlaQfi157tbuQQIoy4bkwkWz29q6A9ur/q/krcS/Otx86z9BdPJds yKvM+puX0szsQYDA4nUT75gsw4ScKX6gyBnJPXvTF+TcjcRwUUMn5+WjkcfI9zZfDjV7+ls2N zXgIIkVb/FmwKsRAQydIERC1EpeOt3Du52hvP7Utdby4zJ6bFmwqy4/NrDpMudh5V4SMygzrU nkxPFXiQolDIW+rl6o8pVLjnXWV99EqxoBmyf8tGiLJC5EEI344H+NHS0OYFkwwURkv3vU5eB 2ZO9jvjQsrw9kCE9AXsW3hG1PZ9BIBsGRKiEFy5KgMk829sH1eH0+tbGr6KMkArfmbrF+1/ce Y4NseOwLDwCvBhBSA1z2VxVaslqNNs0I3cFYBAO+CqBgRq/mCpQUpPiWs6lQLPj4eKF/LdCiF gmxO1d2sLfPdZBAr0qIdEwnUftJS4W2dVKB90TolNydZ1cRUwaSzWuhBQn/3WWWXNwB/Cy23E DOioOGB7BzwCMMPCccWQo2Ezf4kLCKqI9pnqDIy7Td+HiHMeLqnchqJ88leQQCeDM24f+joVp XSCtCragfEsZWKnrsNB4Pk7SBdhM4wwM= +1 on deprecating the contrib folder. Bolk de Bruin the reason the core hooks and operators are properly tested because, for example I added some more tests to it and I am "only" a contributor. So do you really want to split up contributors work and core committers work? I personally think this is not the right way to go. It is true that the contrib hooks and operators have not the same level of code quality but we can do something about it. I am trying to improve our test coverage overall and add missing tests. I don't think an extra step is needed here where we first move properly tested ones into the core package and then moving new ones from time to time into it. Wouldn't that mean that we think the code quality of "contrib" (contributor) in general is worse than the code quality of committers? Every new contributor who comes along this project would think that, wouldn't he? -feluelle Am 13/04/2019 um 07:51 schrieb Beau Barker: > A separate airflow-contrib repo, on a separate release cadence would be = my preference. > > >> On 12 Apr 2019, at 11:17 pm, Julian De Ruiter wrote: >> >> Isn=E2=80=99t this in contradiction with AIP-8, which is aimed at remov= ing operators/hooks from the core Airflow package? >> >> Personally I would rather remove hooks/operators from Airflow than add = even more to the Airflow core. This counts double for the contrib stuff, w= hich is often poorly designed and/or tested. >> >> Best, >> Julian >> >>> On 12 Apr 2019, at 10:23, Bolke de Bruin wrote: >>> >>> That=E2=80=99s perfectly fine to me. >>> >>> Verstuurd vanaf mijn iPad >>> >>>> Op 12 apr. 2019 om 10:20 heeft Kaxil Naik het v= olgende geschreven: >>>> >>>> Ok. How about moving the properly tested and maintained hooks/ops fro= m >>>> contrib to core? >>>> >>>>> On Fri, Apr 12, 2019, 09:13 Bolke de Bruin wrote= : >>>>> >>>>> I disagree. Core signals =E2=80=9Cproperly tested=E2=80=9D and maint= ained. Ie. A kind of >>>>> quality. I don=E2=80=99t think contrib has that. >>>>> >>>>> Verstuurd vanaf mijn iPad >>>>> >>>>>> Op 12 apr. 2019 om 10:03 heeft Kaxil Naik het >>>>> volgende geschreven: >>>>>> Contrib folder was used when it was used at Airbnb. Currently, it d= oesn't >>>>>> make any sense and we have equal responsibility to maintain all the >>>>> hooks, >>>>>> operators, sensors in contrib folder as we do for core. >>>>>> >>>>>> I would suggest to remove contrib folder and move all hooks, ops, a= nd >>>>>> sensors to the core folder. >>>>>> >>>>>> Or reorganize the folder structure similar to what was discussed in= a >>>>>> mailing thread few months ago. >>>>>> >>>>>> Regards, >>>>>> Kaxil