airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <Jarek.Pot...@polidea.com>
Subject Re: [VOTE] Changes in import paths
Date Wed, 31 Jul 2019 08:30:58 GMT
Yep. Agree with Ash on it. There are a number of 'action' operators
specific for cloud providers and these should be our target. The transfer
ones require another AIP (A lot of that already discussed in AIP-8
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303).

J.

Principal Software Engineer
Phone: +48660796129

śr., 31 lip 2019, 10:01 użytkownik Ash Berlin-Taylor <ash@apache.org>
napisał:

> This is a good idea for now. I'm also not overly concerned about these few
> non-cloud examples - FTPtoS3Operator can stay where it is and doesn't have
> to move under 'aws.' to my mind.
>
> Longer term I'd like to go back to making the "transfer/copy/transform"
> operators "composable" so that we can have a single DataCopyOperator, and
> give it a source and destination, and it uses hooks to do the work. This is
> not a small undertaking though to do well and efficiently though.
>
> -ash
>
> > On 30 Jul 2019, at 20:54, Tomasz Urbaszek <tomasz.urbaszek@polidea.com>
> wrote:
> >
> > Maybe we can put all those AtoB operators under one name like “transfer”,
> > then it would be easier to look for such operator?
> >
> > Best,
> > Tomek
> >
> > On Tue, 30 Jul 2019 at 21:39, Chen Tong <cixuuz@gmail.com> wrote:
> >
> >> Daniel mentioned a good point. Such composed operator may also involves
> >> both cloud and non-cloud provider saying FTPtoS3Operator. Should it in
> AWS
> >> folder?
> >>
> >> Also, saying in the future, another cloud provider is growing large
> enough.
> >> Will we rename all plugins related to this provider? What's the
> criteria we
> >> say it's big enough? And option D gives an impression like these tools
> may
> >> be maintained and supported by the Cloud provider. I am not sure if it
> will
> >> be a truth or not.
> >>
> >> Best,
> >>
> >>
> >> On Tue, Jul 30, 2019 at 11:18 AM Daniel Standish <dpstandish@gmail.com>
> >> wrote:
> >>
> >>> One wrinkle with case 3+4 option D is inter-provider operators.  Mainly
> >>> it's storage I think e.g. XToS3Operator or XToGCSOperator where the X
> is
> >> a
> >>> service some different provider.
> >>>
> >>> Maybe the rule should be to locate the operator according to the first
> >>> provider referenced.  So e.g. s3_to_gcs_transfer_operator.py would go
> to
> >>> aws.
> >>>
> >>>
> >>> On Tue, Jul 30, 2019 at 6:21 AM Kamil Breguła <
> kamil.bregula@polidea.com
> >>>
> >>> wrote:
> >>>
> >>>> Yes. All changes will be backwards compatible. In the case of using
> >>>> the old path, a message containing a proposal for change will be
> >>>> reported to the user.
> >>>>
> >>>> I prepared an example of how to change the name of a class in a case
> >>>> with the use of a native solution.
> >>>> Source code:
> >>>>
> >>
> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/rename
> >>>> Preview from IDE: https://imgur.com/a/45NxS5W
> >>>>
> >>>> On Tue, Jul 30, 2019 at 2:28 PM Ash Berlin-Taylor <ash@apache.org>
> >>> wrote:
> >>>>>
> >>>>> Just cos I'm not sure it's _explicitly_ stated, but all of the moves
> >>>> will have a deprecation of the old name right?
> >>>>>
> >>>>> 3+4 case D gets my vote too.
> >>>>>
> >>>>> -a
> >>>>>
> >>>>>> On 30 Jul 2019, at 09:58, Jarek Potiuk <Jarek.Potiuk@polidea.com>
> >>>> wrote:
> >>>>>>
> >>>>>> I went ahead and updated the page (just to speed it up) as I think
> >> it
> >>>>>> really makes sense to join those two cases (and I do not see any
> >>>> drawbacks
> >>>>>> - I think the options we have cover all possible approaches) and we
> >>> can
> >>>>>> always go back if we need to.
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal
> >>>>>>
> >>>>>> My vote is *D*.
> >>>>>>
> >>>>>>  - airflow/contrib/operators/gcp_bigtable_operator.py  →
> >>> airflow/gcp
> >>>>>>  /operators/bigtable_operator.py
> >>>>>>  - airflow/contrib/operators/ssh_operator.py ->
> >>>>>>  airflow/operators/ssh_operator.py
> >>>>>>
> >>>>>> I updated the page with my vote.
> >>>>>> I propose that we vote till Friday on that one (all the rest is
> >>> already
> >>>>>> agreed).
> >>>>>>
> >>>>>> J.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jul 30, 2019 at 9:42 AM Jarek Potiuk <
> >>> Jarek.Potiuk@polidea.com
> >>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I think almost everyone voted and we have almost perfect
> >> consensus.
> >>>> We all
> >>>>>>> agree amongst other on moving all operators out of contrib
> >> (Great!).
> >>>>>>>
> >>>>>>> The only doubts are for *Case 3* (Cloud provider prefix) and *Case
> >>> 4*
> >>>>>>> (Using Namespaces).
> >>>>>>> I think there was actually an overlap between those two. Also
> >> Ash's
> >>>>>>> comment/veto on that is quite clear.
> >>>>>>> So I have final (I hope!) proposal to join those two cases and
> >> have
> >>>> one
> >>>>>>> voting (till Friday) only on that.
> >>>>>>>
> >>>>>>> I will update the doc if you all agree with me that makes more
> >> sense
> >>>> as
> >>>>>>> single case to vote on:
> >>>>>>>
> >>>>>>> *[Case 3 + Case 4]: Grouping Cloud Provider operators.*
> >>>>>>>
> >>>>>>> *Option A*. Keep operators/sensors/hooks in
> >>> airflow/operators(sensors,
> >>>>>>> hooks) and keep/add prefixes in file names.
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py  becomes
> >>>>>>>  airflow/operators/**aws_sns_publish_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> >>>>>>> becomes *airflow/operators/gcp_dataproc_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/hooks/gcp_bigtable_hook.py  *becomes *airflow/*
> >>>>>>>  *hooks/gcp_bigtable_hook.py*
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes *airflow/*
> >>>>>>>  *operators/ssh_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> *Option B*. Group operators/sensors/hooks in
> >>>> airflow/operators(sensors,
> >>>>>>> hooks)/*<PROVIDER>.* Non-cloud provider ones are moved to
> >>>>>>> airflow/operators(sensors/hooks), Drop the prefix.
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> >>>>>>>  becomes airflow/operators/**aws/sns_publish_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> >>>>>>> becomes *airflow/operators/gcp/dataproc_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py  *becomes
> >>>> *airflow/*
> >>>>>>>  *operators/gcp/bigtable_operator.py*
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes *airflow/*
> >>>>>>>  *operators/ssh_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>> *Option C*. Group operators/sensors/hooks in
> >>>> airflow/operators(sensors,
> >>>>>>> hooks)*/<PROVIDER>.* Non-cloud provider ones are moved to
> >>>>>>> airflow/operators(sensors/hooks), Keep the prefix.
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> >>>>>>>  becomes airflow/operators/**aws/aws_sns_publish_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> >>>>>>> becomes *airflow/operators/gcp/gcp_dataproc_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py  *becomes
> >>>> *airflow/*
> >>>>>>>  *operators/gcp/gcp_bigtable_operator.py*
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes *airflow/*
> >>>>>>>  *operators/ssh_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>> *Option D.* Group operators/sensors/hooks in
> >>>> *airflow/<PROVIDER>*/operators(sensors,
> >>>>>>> hooks). Non-cloud provider ones are moved to
> >>>>>>> airflow/operators(sensors/hooks). Drop the prefix.
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> >>>>>>>  becomes airflow/aws/operators/**sns_publish_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> >>>>>>> becomes *airflow/gcp/operators/dataproc_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py  *becomes
> >>>> *airflow/*
> >>>>>>>  *gcp/operators/bigtable_operator.py*
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes *airflow/*
> >>>>>>>  *operators/ssh_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>> *Option E.* Group operators/sensors/hooks in
> >>>> *airflow/<PROVIDER>*/operators(sensors,
> >>>>>>> hooks). Non-cloud provider ones are moved to
> >>>>>>> airflow/operators(sensors/hooks).* Keep the prefix*.
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> >>>>>>>  becomes airflow/aws/operators/aws_**sns_publish_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> >>>>>>> becomes *airflow/gcp/operators/gcp_dataproc_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py  *becomes
> >>>> *airflow/*
> >>>>>>>  *gcp/operators/gcp_bigtable_operator.py*
> >>>>>>>  -
> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes *airflow/*
> >>>>>>>  *operators/ssh_operator.py*
> >>>>>>>
> >>>>>>>
> >>>>>>> Shall I proceed with updating the page ?
> >>>>>>>
> >>>>>>>
> >>>>>>> J.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jul 29, 2019 at 11:18 AM Kaxil Naik <kaxilnaik@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Thanks Jarek, adding my vote.
> >>>>>>>>
> >>>>>>>> On Mon, Jul 29, 2019 at 2:40 PM Ash Berlin-Taylor <
> >> ash@apache.org>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>>> Thanks Jarek, much clearer. I have voted there too.
> >>>>>>>>>
> >>>>>>>>> -ash
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On 29 Jul 2019, at 08:13, Driesprong, Fokko
> >> <fokko@driesprong.frl
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Thanks Jarek, the Wiki works much better for me. I've added my
> >>> vote
> >>>>>>>> there
> >>>>>>>>>> as well.
> >>>>>>>>>>
> >>>>>>>>>> You've convinced me on the second case. I've changed my vote on
> >>>> Case 2
> >>>>>>>>> from
> >>>>>>>>>> A to B :-)
> >>>>>>>>>>
> >>>>>>>>>> Maybe we should leave the vote open a bit longer since it
> >>> involves
> >>>>>>>> quite
> >>>>>>>>>> some reading, and I would like to get some proper feedback and
> >>>>>>>> opinions
> >>>>>>>>> on
> >>>>>>>>>> this. Plus I only see two votes right now. If we don't think
> >> this
> >>>>>>>>> through,
> >>>>>>>>>> it might be that we're having this vote again in the near
> >> future
> >>>> :-)
> >>>>>>>>>>
> >>>>>>>>>> Cheers, Fokko
> >>>>>>>>>>
> >>>>>>>>>> Op zo 28 jul. 2019 om 10:49 schreef Jarek Potiuk <
> >>>>>>>>> Jarek.Potiuk@polidea.com>:
> >>>>>>>>>>
> >>>>>>>>>>> I updated the AIP-21 proposal in the way that should answer
> >> all
> >>>> the
> >>>>>>>>>>> concerns in this thread.
> >>>>>>>>>>>
> >>>>>>>>>>> NOTE! There is an action for all of those who are interested:
> >>>> Please
> >>>>>>>>>>> fill-in your voting in the voting table till Tuesday 30th of
> >>> July
> >>>> 6pm
> >>>>>>>>> CEST
> >>>>>>>>>>> (5pm BST, 9 am PST).
> >>>>>>>>>>>
> >>>>>>>>>>> I created a summary of the proposal
> >>>>>>>>>>> <
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal
> >>>>>>>>>>>>
> >>>>>>>>>>> in table form - which contains also examples for each case.
> >>>>>>>>>>>
> >>>>>>>>>>> I added a voting table
> >>>>>>>>>>> <
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting
> >>>>>>>>>>>>
> >>>>>>>>>>> where you should add your votes:
> >>>>>>>>>>>
> >>>>>>>>>>> I propose this voting mechanism:
> >>>>>>>>>>>
> >>>>>>>>>>> Voting will take place till *Tuesday* 30 Jul 2019 6pm CEST  (5
> >>> pm
> >>>>>>>> BST,
> >>>>>>>>> 9am
> >>>>>>>>>>> PST) .
> >>>>>>>>>>>
> >>>>>>>>>>> After the choice, the final consistent set of choices will be
> >>>>>>>> announced
> >>>>>>>>>>> (taking into account majority of binding vote, also including
> >>>>>>>> potential
> >>>>>>>>>>> vetos and consistency between the choices. Non-binding votes
> >>> will
> >>>> be
> >>>>>>>>> taken
> >>>>>>>>>>> into account in case there is a draw. The final set of choices
> >>>> will
> >>>>>>>> be
> >>>>>>>>>>> announced at devlist thread
> >>>>>>>>>>> <
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://lists.apache.org/thread.html/4cedc94bee53ad908eee8333a56b58be8b5641881e73f69b97e436a9@%3Cdev.airflow.apache.org%3E
> >>>>>>>>>>>>
> >>>>>>>>>>> after
> >>>>>>>>>>> the voting completes.
> >>>>>>>>>>>
> >>>>>>>>>>> Unless there is a veto raised to the final proposal, the final
> >>>>>>>> proposal
> >>>>>>>>> is
> >>>>>>>>>>> accepted by Lazy Consensus
> >>>>>>>>>>> <https://community.apache.org/committers/lazyConsensus.html>
> >>> on
> >>>>>>>>> *Friday*
> >>>>>>>>>>> 02
> >>>>>>>>>>> Aug 2019 at 6pm CEST (5 pm BST, 9am PST).
> >>>>>>>>>>>
> >>>>>>>>>>> Please let me know if what I proposed can be further improved
> >> to
> >>>>>>>> avoid
> >>>>>>>>>>> chaos.
> >>>>>>>>>>>
> >>>>>>>>>>> J.
> >>>>>>>>>>>
> >>>>>>>>>>> On Sat, Jul 27, 2019 at 11:41 AM Jarek Potiuk <
> >>>>>>>> Jarek.Potiuk@polidea.com
> >>>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Ok. I see that indeed voting could have been organised a bit
> >>>> better
> >>>>>>>> -
> >>>>>>>>> dev
> >>>>>>>>>>>> list does not seem to cope well with such a complex case :).
> >>>>>>>>>>>>
> >>>>>>>>>>>> As Kamil noticed - we already have a formal AIP-21
> >>>>>>>>>>>> <
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths
> >>>>>>>>>>>>
> >>>>>>>>>>>> in the Wiki - which I should have mentioned before. I have
> >> not
> >>>> much
> >>>>>>>>> time
> >>>>>>>>>>>> today - but tomorrow I will try to summarise the options -
> >>> based
> >>>> on
> >>>>>>>>> some
> >>>>>>>>>>>> real code examples (to make it easier to digest). I think the
> >>>> best
> >>>>>>>>>>> approach
> >>>>>>>>>>>> will be to make a voting matrix in the AIP-21 Wiki page where
> >>>> people
> >>>>>>>>> will
> >>>>>>>>>>>> be able to state their preferences for each case (by adding a
> >>>> row to
> >>>>>>>>> the
> >>>>>>>>>>>> table). I think that might provide much better clarity and a
> >>>> single
> >>>>>>>>> place
> >>>>>>>>>>>> which will show the current aggregated state of voting.
> >>>>>>>>>>>>
> >>>>>>>>>>>> However - as Ash also stated - some of the options are
> >>>>>>>> inter-dependent
> >>>>>>>>> so
> >>>>>>>>>>>> at the end we will have to adjust the results in case they
> >> are
> >>>> not
> >>>>>>>>>>>> consistent  - and make final voting on aggregated set of
> >> cases
> >>> -
> >>>>>>>> but I
> >>>>>>>>>>>> think in this case following "lasy consensus" - i,e. if noone
> >>>>>>>> objects
> >>>>>>>>> to
> >>>>>>>>>>>> final set of cases, we will proceed with it..
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you think that will work better ?
> >>>>>>>>>>>>
> >>>>>>>>>>>> J.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Principal Software Engineer
> >>>>>>>>>>>> Phone: +48660796129
> >>>>>>>>>>>>
> >>>>>>>>>>>> sob., 27 lip 2019, 09:26 użytkownik Kamil Breguła <
> >>>>>>>>>>>> kamil.bregula@polidea.com> napisał:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Document is also available as AIP on wiki:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sat, Jul 27, 2019, 9:07 AM Driesprong, Fokko
> >>>>>>>> <fokko@driesprong.frl
> >>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I agree with all, this is a bit chaotic. For the sake of
> >>>> clarity,
> >>>>>>>> I
> >>>>>>>>>>>>> would
> >>>>>>>>>>>>>> suggest moving the Google Doc to the Wiki as an AIP.
> >> Create a
> >>>>>>>>>>> structural
> >>>>>>>>>>>>>> code improvement AIP and then and add a matrix of
> >> users/cases
> >>>>>>>> where
> >>>>>>>>>>>>>> everybody can add his vote per case. This gives also the
> >>>>>>>> opportunity
> >>>>>>>>>>> for
> >>>>>>>>>>>>>> changing your vote on a specific case. WDYT?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cheers, Fokko
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Op vr 26 jul. 2019 om 22:28 schreef Chen Tong <
> >>>> cixuuz@gmail.com>:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I agree with Ash. It is clear to vote on each case rather
> >>> than
> >>>>>>>> all
> >>>>>>>>>>>>>>> together.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:54 PM Ash Berlin-Taylor <
> >>>>>>>> ash@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> A number of proposals overlap or add on to each other,
> >> so I
> >>>>>>>> think
> >>>>>>>>>>>>> (?) a
> >>>>>>>>>>>>>>>> single vote makes sense.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> To be clear, my vote is -1 until it's clear exactly what
> >> we
> >>>> are
> >>>>>>>>>>>>> voting
> >>>>>>>>>>>>>> on.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 26 July 2019 20:39:56 BST, Kaxil Naik <
> >>>> kaxilnaik@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>> I agree with Ash and instead of voting on all 7 Cases
> >>>> together,
> >>>>>>>>>>> how
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>> separate vote emails for all the cases? Each email can
> >>> have
> >>>> the
> >>>>>>>>>>>>>>>>> description
> >>>>>>>>>>>>>>>>> copied over from the doc.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It would probably be a bit easier to track a decision in
> >>> the
> >>>>>>>>>>> future
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> What do you guys think? @Jarek Potiuk <
> >>>>>>>> Jarek.Potiuk@polidea.com>
> >>>>>>>>>>>>>>>>> @Driesprong,
> >>>>>>>>>>>>>>>>> Fokko <fokko@driesprong.frl>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Sat, Jul 27, 2019 at 1:03 AM Kaxil Naik <
> >>>>>>>> kaxilnaik@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Case #1* airflow.contrib.{resources} : *Solution A *
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Case #2* git *_{operator/sensor}{/s}.py : *Solution A
> >> *
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Case #3* {aws/azure/gcp}_* : *Solution C*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Case #4* Separate namespace for resources :
> >>>>>>>>>>>>>>>>>> *Solution A*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Case #5* *Operator : *Solution A*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Case #7* : *Solution A*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 11:09 PM Daniel Standish
> >>>>>>>>>>>>>>>>> <dpstandish@gmail.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I have tried to add some clarification to Jarek's
> >>> summary,
> >>>>>>>> but
> >>>>>>>>>>> I
> >>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> little fuzzy on exact proposal for case 3 so hopefully
> >>>> Jarek
> >>>>>>>>>>> can
> >>>>>>>>>>>>>>>>> further
> >>>>>>>>>>>>>>>>>>> clarify.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> According to my reading, in this motion cases 4,5,6
> >> are
> >>>> all
> >>>>>>>>>>>>> either
> >>>>>>>>>>>>>>>>>>> proposal
> >>>>>>>>>>>>>>>>>>> *rejections* or otherwise have no effect, so attention
> >>>> can be
> >>>>>>>>>>>>>>>>> focused on
> >>>>>>>>>>>>>>>>>>> cases 1,2,3 and 7.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> *Motion is to adopt the following:*
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 1 - Solution A - Get rid of contrib
> >>>>>>>>>>>>>>>>>>> All objects will be moved out of contrib folder and
> >> into
> >>>>>>>>>>>>> respective
> >>>>>>>>>>>>>>>>>>> non-contrib folder.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 2 - Solution A - Remove object type suffix from
> >>>> modules
> >>>>>>>>>>>>>>>>>>> Example:
> >>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator becomes
> >>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator
> >>>>>>>>>>>>>>>>>>> Example:
> >>>>>>>>>>>>>>>>>>> Airflow.hooks.foo_hook.FooOperator becomes
> >>>>>>>>>>>>> airflow.hooks.foo.FooHook
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 3 - conventions for cloud provider etc prefixes
> >>>>>>>>>>>>>>>>>>> *I am not sure exactly what the proposal is.*
> >>>>>>>>>>>>>>>>>>> Example case is
> >>>>>>>>>>>>> airflow/contrib/operators/gcp_bigtable_operator.py
> >>>>>>>>>>>>>>>>>>> Since motion adopts case 1 solution A, we can assume
> >> it
> >>> is
> >>>>>>>> now
> >>>>>>>>>>>>> named
> >>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>> so: airflow/operators/gcp_bigtable_operator.py
> >>>>>>>>>>>>>>>>>>> So what is proposed for this case?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 4 - Solution B - *no change*
> >>>>>>>>>>>>>>>>>>> This proposal was about namespaces but the motion is
> >> to
> >>>>>>>> reject
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> proposal.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 5 - Solution B - *no change*
> >>>>>>>>>>>>>>>>>>> The proposal was to remove class type suffix e.g.
> >> remove
> >>>> the
> >>>>>>>>>>>>>>>>> Operator in
> >>>>>>>>>>>>>>>>>>> FooOperator, but the motion is to reject this proposal
> >>> --
> >>>>>>>> i.e.
> >>>>>>>>>>>>>>>>> motion is
> >>>>>>>>>>>>>>>>>>> no
> >>>>>>>>>>>>>>>>>>> change on this case, i.e. we resolve to keep
> >> "Operator"
> >>>> (and
> >>>>>>>>>>>>>>>>> "Sensor")
> >>>>>>>>>>>>>>>>>>> suffixes on class names
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 6 - *No change*
> >>>>>>>>>>>>>>>>>>> This case concerns object naming inconsistencies, but
> >>>> motion
> >>>>>>>>>>> does
> >>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> enact
> >>>>>>>>>>>>>>>>>>> any specific proposal.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Case 7 - Solution A - use warnings.warn template for
> >>>>>>>>>>> deprecation
> >>>>>>>>>>>>>>>>> warnings
> >>>>>>>>>>>>>>>>>>> (instead of zope)
> >>>>>>>>>>>>>>>>>>> Example:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> # in old_package/old_mod.py
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> import warnings
> >>>>>>>>>>>>>>>>>>> from new_package.new_mod import *
> >>>>>>>>>>>>>>>>>>> warnings.warn("old_package.old_mod has moved to
> >>>>>>>>>>>>> new_package.new_mod.
> >>>>>>>>>>>>>>>>>>> Import
> >>>>>>>>>>>>>>>>>>> of "
> >>>>>>>>>>>>>>>>>>> "old_package.old_mod will become unsupported in
> >> version
> >>>> 2",
> >>>>>>>>>>>>>>>>>>> DeprecationWarning, 2)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Reference implementation here:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/solution1
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> FWIW +1 (non-binding) on 1,2,7 -- unsure on 3.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I am very happy that this motion now gets rid of
> >>> contrib.
> >>>>>>>>>>> Thank
> >>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> Sergio
> >>>>>>>>>>>>>>>>>>> for turning the tide with your effective argumentation
> >>> ;)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:16 AM Ash Berlin-Taylor <
> >>>>>>>>>>>>> ash@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Could someone summarise what the proposed name
> >> changes
> >>>> are,
> >>>>>>>>>>>>> here,
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>> thread? Pointing at a google doc and only giving
> >>> partial
> >>>>>>>>>>>>> examples
> >>>>>>>>>>>>>>>>> is 1)
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> bit difficult to quickly work out what we are voting
> >>> on,
> >>>> and
> >>>>>>>>>>> 2)
> >>>>>>>>>>>>>>>>> using a
> >>>>>>>>>>>>>>>>>>>> google doc isn't great for a longer term record.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> Ash
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On 26 Jul 2019, at 09:14, Jarek Potiuk
> >>>>>>>>>>>>>>>>> <Jarek.Potiuk@polidea.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I would like to recast the vote. Let's start from 0
> >> :)
> >>>> . I
> >>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> keep the July 30th 6pm CEST date, and at least three
> >>> +1
> >>>>>>>>>>>>>>>>> (binding)
> >>>>>>>>>>>>>>>>>>> votes
> >>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>> needed to pass. I marked in red the modifications to
> >>> the
> >>>>>>>>>>>>>>>>> previous
> >>>>>>>>>>>>>>>>>>>> proposal.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Here is the proposal (details here
> >>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#heading=h.j4jc3i70qszo
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> )
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> - *Case 1: A: Get rid of contrib,*
> >>>>>>>>>>>>>>>>>>>>> - *Case 2: A:
> >>> Airflow.operators.foo_operator.FooOperator
> >>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator*
> >>>>>>>>>>>>>>>>>>>>> - *Case 3: C:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator
> >>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>> become
> >>> airflow.gcp.operators.bigtable.BigTableOperator*
> >>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction*
> >>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor")
> >> suffixes
> >>> on
> >>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>> names*
> >>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on
> >>> case-by-case
> >>>>>>>>>>>>> (and
> >>>>>>>>>>>>>>>>> my team
> >>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)*
> >>>>>>>>>>>>>>>>>>>>> - *Case 7: Native python solution (with better IDE
> >>>>>>>>>>>>>>>>> integration)*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> This is my binding (+1) vote.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 10:08 AM Jarek Potiuk <
> >>>>>>>>>>>>>>>>>>> Jarek.Potiuk@polidea.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hey Felix,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> For 7* -> I am in favour of trying the native one
> >> as
> >>>> well
> >>>>>>>>>>> (I
> >>>>>>>>>>>>>>>>> use IDE
> >>>>>>>>>>>>>>>>>>>> quite
> >>>>>>>>>>>>>>>>>>>>>> often ;) )
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> J.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 24, 2019 at 9:34 AM Driesprong, Fokko
> >>>>>>>>>>>>>>>>>>> <fokko@driesprong.frl
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks Kamil for putting the document together. I
> >>>> wasn't
> >>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> respond
> >>>>>>>>>>>>>>>>>>>>>>> earlier since I was giving Airflow workshops
> >>>> throughout
> >>>>>>>>>>>>> Europe
> >>>>>>>>>>>>>>>>> :-)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *Case 1: *Solution A → Remove everything from
> >>> contrib
> >>>>>>>>>>> into
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> single
> >>>>>>>>>>>>>>>>>>>>>>> package.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> To make it easier to the end-user, my preference
> >>>> would be
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>> rid of
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> contrib package altogether. What's in contrib or
> >> not
> >>>> is a
> >>>>>>>>>>>>>>>>> tricky
> >>>>>>>>>>>>>>>>>>>> question,
> >>>>>>>>>>>>>>>>>>>>>>> I think this is already reflected in the document
> >>>> since
> >>>>>>>>>>>>>>>>> "maturity"
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> quotes. What would be the moment to transfer the
> >>>> operator
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> core
> >>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>> vice versa? I'm afraid that renaming contrib to
> >>>>>>>>>>> incubating
> >>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>> confuse
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> user even more. For people who aren't as known
> >> with
> >>>>>>>>>>>>> Airflow it
> >>>>>>>>>>>>>>>>> isn't
> >>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> transparent which operator lives where, especially
> >>> if
> >>>> you
> >>>>>>>>>>>>>>>>> don't use
> >>>>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>> IDE
> >>>>>>>>>>>>>>>>>>>>>>> such as Ash ;) Besides that I don't think it is
> >>> worth
> >>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> public
> >>>>>>>>>>>>>>>>>>>>>>> API just for a different name.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Ok. I am convinced. I will re-cast the vote with
> >>> this.
> >>>> It
> >>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>> easier
> >>>>>>>>>>>>>>>>>>>>>> as we will not have to define other rules as those
> >>>> that we
> >>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>> have.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *Case 2:* Slight preference to Solution B (let's
> >>> say a
> >>>>>>>>>>> +0)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I like to search on Github with t, and then search
> >>> for
> >>>>>>>>>>>>>>>>> BashOperator.
> >>>>>>>>>>>>>>>>>>>> After
> >>>>>>>>>>>>>>>>>>>>>>> the change, you should search for Operator/Bash.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Jarek, I'm confused with your vote on this, from
> >>> your
> >>>>>>>>>>> mail
> >>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>> state: -
> >>>>>>>>>>>>>>>>>>>>>>> *Case 2: B:
> >>> Airflow.operators.foo_operator.FooOperator
> >>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>> become
> >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator*. But the B
> >>>> solution is
> >>>>>>>>>>>>>>>>> keeping it
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> same, so that would mean - *Case 2: B:
> >>>>>>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator *would
> >>> stay
> >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo_operator*.FooOperator*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> My mistake. It was of course A. I think there is a
> >>>> little
> >>>>>>>>>>>>>>>>> confusion
> >>>>>>>>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>> This case is not for changing BashOperator into
> >> Bash
> >>>> but
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> changing
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> *module* name not the *class* name. So
> >>>>>>>>>>>>>>>>> bash_operator.BashOperator ->
> >>>>>>>>>>>>>>>>>>>>>> bash.BashOperator :) . you will still search for
> >>>>>>>>>>>>> BashOperator
> >>>>>>>>>>>>>>>>> in this
> >>>>>>>>>>>>>>>>>>>> case.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *Case 3:* I'm leaning towards B
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Mostly because it isn't as trivial to decide when
> >> it
> >>>> gets
> >>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>> own
> >>>>>>>>>>>>>>>>>>>> package
> >>>>>>>>>>>>>>>>>>>>>>> or not. Besides the cloud providers, there are
> >>>> companies
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>> Databricks
> >>>>>>>>>>>>>>>>>>>>>>> and Qubole which might also end up with their own
> >>>> package
> >>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>> their
> >>>>>>>>>>>>>>>>>>>>>>> integration is getting richer over time. Moving
> >> this
> >>>> is a
> >>>>>>>>>>>>> bit
> >>>>>>>>>>>>>>>>>>> painful
> >>>>>>>>>>>>>>>>>>>>>>> because we need to deprecate the old path and move
> >>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>> introduces noise into version control etc.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I'd say, we should go for C. As I proposed. It's
> >> not
> >>> as
> >>>>>>>>>>>>>>>>> difficult as
> >>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> seems to group operators in packages and it has
> >>>> numerous
> >>>>>>>>>>>>>>>>> advantages.
> >>>>>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>> nice thing about deprecations - we can do them in
> >> the
> >>>>>>>>>>>>>>>>> __init__.py of
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> packages which causes the noise fairly small (Git
> >>> will
> >>>>>>>>>>>>> actually
> >>>>>>>>>>>>>>>>>>> realise
> >>>>>>>>>>>>>>>>>>>>>> that the file has been moved and you can follow
> >> that.
> >>>>>>>>>>> Maybe
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>> super
> >>>>>>>>>>>>>>>>>>>>>> easy to merge changes later to 1.10.4  but it's
> >> not a
> >>>>>>>>>>> rocket
> >>>>>>>>>>>>>>>>> science
> >>>>>>>>>>>>>>>>>>>> either.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *Case 7:* Python native solution
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Yes.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Apart from all, great work!
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Cheers, Fokko
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Op di 23 jul. 2019 om 21:27 schreef Felix
> >> Uellendall
> >>>>>>>>>>>>>>>>>>>>>>> <feluelle@pm.me.invalid
> >>>>>>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I have no strong "No" against any proposed change
> >>> of
> >>>>>>>>>>> these
> >>>>>>>>>>>>>>>>> cases.
> >>>>>>>>>>>>>>>>>>> So I
> >>>>>>>>>>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>>> with +1 (non-binding).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> P.S. Thanks Jarek for bringing this up again and
> >>> your
> >>>>>>>>>>>>> intense
> >>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>> towards
> >>>>>>>>>>>>>>>>>>>>>>>> airflow currently :) and thanks to Kamil for even
> >>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> document. I
> >>>>>>>>>>>>>>>>>>>>>>>> like how the code is getting more and more
> >>> consistent
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> clean :)
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Kind regards,
> >>>>>>>>>>>>>>>>>>>>>>>> Felix
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Sent from ProtonMail mobile
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -------- Original Message --------
> >>>>>>>>>>>>>>>>>>>>>>>> On Jul 23, 2019, 18:34, Jarek Potiuk wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> This email is calling a vote on the changes in
> >>>> import
> >>>>>>>>>>>>> paths.
> >>>>>>>>>>>>>>>>> It's
> >>>>>>>>>>>>>>>>>>>> been
> >>>>>>>>>>>>>>>>>>>>>>>>> discussed in
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://lists.apache.org/thread.html/4e648d9421c792d4537f5ac66f1a16dce468f816fc5221a9f9db9433@%3Cdev.airflow.apache.org%3E
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The vote will last for at least 1 week (July
> >> 30th
> >>>> 6pm
> >>>>>>>>>>>>> CEST),
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>>> least
> >>>>>>>>>>>>>>>>>>>>>>>>> three +1 (binding) votes have been cast.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The proposal to vote is based on the document
> >> from
> >>>>>>>>>>> Kamil
> >>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The proposed solution is:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 1: B: Contrib vs not: we move all that
> >> are
> >>>>>>>>>>> "well"
> >>>>>>>>>>>>>>>>> tested
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> rename contrib to "incubating" or similar.*
> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 2: B:
> >>>>>>>>>>> Airflow.operators.foo_operator.FooOperator
> >>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator*
> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 3: C:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator
> >>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>> become
> >>>> airflow.gcp.operators.bigtable.BigTableOperator*
> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction*
> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor")
> >>>> suffixes
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>> names*
> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on
> >>>> case-by-case
> >>>>>>>>>>>>> (and
> >>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>> team
> >>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> This is my (binding) +1 vote.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> J.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk
> >>>>>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal
> >>>>>>>>>>> Software
> >>>>>>>>>>>>>>>>> Engineer
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> M: [+48 660 796 129](tel:+48660796129)
> >>>>>>>>>>>>>>>>>>>>>>> <[+48660796129](tel:+48660796129)>
> >>>>>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk
> >>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal
> >>>> Software
> >>>>>>>>>>>>>>>>> Engineer
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129>
> >>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Jarek Potiuk
> >>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal
> >>> Software
> >>>>>>>>>>>>>> Engineer
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129>
> >>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> *Kaxil Naik*
> >>>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer*
> >>>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer | *Certified*
> >>> Apache
> >>>>>>>>>>> Spark
> >>>>>>>>>>>>> &
> >>>>>>>>>>>>>>>>> Neo4j
> >>>>>>>>>>>>>>>>>> Developer
> >>>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Kaxil Naik*
> >>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer*
> >>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer | *Certified*
> >>> Apache
> >>>>>>>> Spark
> >>>>>>>>>>> &
> >>>>>>>>>>>>>>>>> Neo4j
> >>>>>>>>>>>>>>>>> Developer
> >>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>> Jarek Potiuk
> >>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software
> >>> Engineer
> >>>>>>>>>>>
> >>>>>>>>>>> M: +48 660 796 129 <+48660796129>
> >>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Kaxil Naik*
> >>>>>>>> *Big Data Consultant | DevOps Data Engineer*
> >>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache
> >> Spark &
> >>>> Neo4j
> >>>>>>>> Developer
> >>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>> Jarek Potiuk
> >>>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>>>>>>
> >>>>>>> M: +48 660 796 129 <+48660796129>
> >>>>>>> [image: Polidea] <https://www.polidea.com/>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>>
> >>>>>> Jarek Potiuk
> >>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>>>>>
> >>>>>> M: +48 660 796 129 <+48660796129>
> >>>>>> [image: Polidea] <https://www.polidea.com/>
> >>>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message