airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Driesprong, Fokko" <fo...@driesprong.frl>
Subject Re: [VOTE] Changes in import paths
Date Mon, 05 Aug 2019 18:34:53 GMT
Hi Jarek,

No worries. I must admit that I've lost track a bit when the case 3+4 got
combined. Maybe for the future better to let the current vote run and maybe
reinitiate a new vote if there are new options.

Currently, it is quite easy to think of a few obvious groups, but it is so
hard to keep this consistent in the future. I'm not saying that one giant
operator package is ideal, but I think it is easy to maintain. Two/three
years ago the guys for Databricks wrote all the operators for their
platform, and it was in such a state that it was eligible to move to the
core (non-contrib), similar with GCP now. But it is hard to actually do it.
Putting everything into one place makes this from my perspective simpler.
If Ali-Cloud comes with some operators, should we put them in a separate
package right away? Prefixing would be a nice compromise for me.

If I'm the only one against adding the additional namespaces, then I don't
want to stop the rest of the community of making this happen. I don't like
to veto stuff, but I think we should think this through properly :-)

Cheers, Fokko


Op ma 5 aug. 2019 om 17:55 schreef Jarek Potiuk <Jarek.Potiuk@polidea.com>:

> Hey Fokko,
>
> Really sorry, to miss this . I see now that you have not voted because you
> thought it will be transferred automatically -it was thought as not and
> additional option but  combined replacement for (3+4) cases.
>
> This was after I realised that there was a veto for namespaces from Ash and
> I realised from it that namespaces were a bit confusing and overlapping
> with grouping per-cloud-provider (see the voting "chapter
> <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting
> >").
> I re-fined the choice back then - whether to not group them (A) or whether
> to group first by type of entity or first by cloud provider (B-E in various
> ways). Maybe I was not very clear in my message - apologies.
>
> I now re-added your vote and also Ben's vote who also voted on leaving the
> prefix and not grouping operators (Ben please correct me if I was wrong).
> Still the result is 4:2 (binding), 6:2 (including non-binding) in favour of
> grouping per cloud provider rather than using prefixes. And no vetoes.
>
> Re - the point of which operators should be grouped. From the discussions -
> I do not think it's easy to do a general and always applicable rule. But I
> believe we agreed that we group the "obvious" choices - like "gcp",
> "azure", "aws". It's for pure "Action" type operators - but still we leave
> the cross-platform "transfer" ones or non-obvious ones in the old place
> (which we keep as bag of "everything else"). Personally - I am all for
> grouping the qubole/databricks operators in packages same as other cloud
> providers. But with the "softness" of that rule this might (and should) be
> an individual decision of people who maintain it. And it is quite OK - not
> everything has to be decided upfront. I don't think we have a "hard" rule
> now that covers all the cases and binds us. It would be very complex one if
> we have it (and we can defer introducing it for later if needed especially
> if it requires more than just renaming/moving but also architectural
> changes - as Ash proposed for the transfer operators). But at least we can
> decrease an entropy by grouping the obvious cases, and we can move forward
> leaving the Airflow world a bit better, one step at a time.
>
> J.
>
> On Mon, Aug 5, 2019 at 3:58 PM Driesprong, Fokko <fokko@driesprong.frl>
> wrote:
>
> > Hi Jarek,
> >
> > Just wondering how the vote is unanimous. On case 3 I've voted for B. I
> > still believe introducing namespaces is hard to maintain, as I explained
> > earlier: Mostly because it isn't as trivial to decide when it gets its
> own
> > package or not. Besides the cloud providers, there are companies like
> > Databricks and Qubole which might also end up with their own package
> > because their integration is getting richer over time. Moving this is a
> bit
> > painful because we need to deprecate the old path and move everything
> which
> > introduces noise into version control etc.
> >
> > Adding an additional option to the list, does not invalidate the older
> > options, right?
> >
> > Cheers, Fokko
> >
> >
> > Op ma 5 aug. 2019 om 15:43 schreef Jarek Potiuk <
> Jarek.Potiuk@polidea.com
> > >:
> >
> > > Seems that after this months of discussion we got an unanimous voting
> > > result. I summarised it in the AIP-21
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths
> > > >
> > > and
> > > changed its status to "Accepted". Thanks for all your feedback! We will
> > > start soon moving GCP operators following those rules and we will
> resolve
> > > any initial problems with those - then we might provide some additional
> > > learnings and see if we can easily (probably semi-automatically) move
> all
> > > other operators.
> > >
> > > Case 1
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#1airflow.contrib.{resources}
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D
> > >
> > > >
> > >
> > > What to do with the "contrib" folder
> > >
> > > Case 2
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#2git*_{operator/sensor}{/s}.py
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py
> > >
> > > >
> > >
> > > Drop modules *_operator suffix
> > >
> > > Case 3
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#3{aws/azure/gcp}_*
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_*>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_*
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_*
> > >
> > > >
> > >  + Case 4
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#4Separatenamespaceforresources
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources
> > >
> > > >
> > >
> > > Grouping Cloud Providers operators/sensors/hooks
> > >
> > > Case 5
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#5*Operator
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator
> > >
> > > >
> > >
> > > *Operator *Sensor *Hook in class name
> > >
> > > Case 6
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#6Otherisolatedcases
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases
> > >
> > > >
> > >
> > > Isolated cases
> > >
> > > Case 7
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#7
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237>
> > <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237
> >
> > > <
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237
> > >
> > > >
> > >
> > > Deprecation method
> > >
> > > *A.* Move everything "contrib" → "airflow"
> > >
> > > *A.* Drop the suffix.
> > >
> > > Example:
> > >
> > >    - *airflow.operator.gcp_bigtable_operator.py*
> > >    becomes *airflow.operator.gcp_bigtable.py
> > >    <http://airflow.operator.gcp_bigtable.py>*.
> > >
> > > *D. * Group operators/sensors/hooks in
> > > *airflow/<PROVIDER>*/operators(sensors,
> > > hooks). Non-cloud provider ones are moved to
> > > airflow/operators(sensors/hooks).
> > > *Drop the prefix.*
> > >
> > >    -
> > > *airflow/contrib/operators/sns_publish_operator.py
> > >    becomes airflow/aws/operators/**sns_publish_operator.py*
> > >
> > >
> > >    - *airflow/contrib/operators/dataproc_operator.py*
> > >   becomes *airflow/gcp/operators/dataproc_operator.py*
> > >
> > >
> > >
> > >    -
> > > *airflow/contrib/operators/gcp_bigtable_operator.py  *becomes
> *airflow/*
> > >    *gcp/operators/bigtable_operator.py*
> > >
> > >
> > >
> > >    -
> > > *airflow/contrib/operators/ssh_operator.py  *becomes *airflow/*
> > >    *operators/ssh_operator.py*
> > >
> > > *B.* No change - keep Operator/Sensor suffix in class name.
> > >
> > > *A.* Make individual decisions of renames for operators that do not
> > follow
> > > common conventions used for other operators.
> > >
> > > Consistency trumps compatibility.
> > >
> > > Examples:
> > >
> > > *DataProcHadoopOperator*
> > >
> > > renamed to:
> > >
> > > *DataprocHadoopOperator*
> > > *A.* Native python method (with better IDE support  and more flexible
> > but a
> > > bit more verbose)
> > >
> > > J.
> > >
> > > On Wed, Jul 31, 2019 at 10:30 AM Jarek Potiuk <
> Jarek.Potiuk@polidea.com>
> > > wrote:
> > >
> > > > Yep. Agree with Ash on it. There are a number of 'action' operators
> > > > specific for cloud providers and these should be our target. The
> > transfer
> > > > ones require another AIP (A lot of that already discussed in AIP-8
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303
> > > > ).
> > > >
> > > > J.
> > > >
> > > > Principal Software Engineer
> > > > Phone: +48660796129
> > > >
> > > > śr., 31 lip 2019, 10:01 użytkownik Ash Berlin-Taylor <ash@apache.org
> >
> > > > napisał:
> > > >
> > > >> This is a good idea for now. I'm also not overly concerned about
> these
> > > >> few non-cloud examples - FTPtoS3Operator can stay where it is and
> > > doesn't
> > > >> have to move under 'aws.' to my mind.
> > > >>
> > > >> Longer term I'd like to go back to making the
> > "transfer/copy/transform"
> > > >> operators "composable" so that we can have a single
> DataCopyOperator,
> > > and
> > > >> give it a source and destination, and it uses hooks to do the work.
> > > This is
> > > >> not a small undertaking though to do well and efficiently though.
> > > >>
> > > >> -ash
> > > >>
> > > >> > On 30 Jul 2019, at 20:54, Tomasz Urbaszek <
> > > tomasz.urbaszek@polidea.com>
> > > >> wrote:
> > > >> >
> > > >> > Maybe we can put all those AtoB operators under one name like
> > > >> “transfer”,
> > > >> > then it would be easier to look for such operator?
> > > >> >
> > > >> > Best,
> > > >> > Tomek
> > > >> >
> > > >> > On Tue, 30 Jul 2019 at 21:39, Chen Tong <cixuuz@gmail.com> wrote:
> > > >> >
> > > >> >> Daniel mentioned a good point. Such composed operator may also
> > > involves
> > > >> >> both cloud and non-cloud provider saying FTPtoS3Operator. Should
> it
> > > in
> > > >> AWS
> > > >> >> folder?
> > > >> >>
> > > >> >> Also, saying in the future, another cloud provider is growing
> large
> > > >> enough.
> > > >> >> Will we rename all plugins related to this provider? What's the
> > > >> criteria we
> > > >> >> say it's big enough? And option D gives an impression like these
> > > tools
> > > >> may
> > > >> >> be maintained and supported by the Cloud provider. I am not sure
> if
> > > it
> > > >> will
> > > >> >> be a truth or not.
> > > >> >>
> > > >> >> Best,
> > > >> >>
> > > >> >>
> > > >> >> On Tue, Jul 30, 2019 at 11:18 AM Daniel Standish <
> > > dpstandish@gmail.com
> > > >> >
> > > >> >> wrote:
> > > >> >>
> > > >> >>> One wrinkle with case 3+4 option D is inter-provider operators.
> > > >> Mainly
> > > >> >>> it's storage I think e.g. XToS3Operator or XToGCSOperator where
> > the
> > > X
> > > >> is
> > > >> >> a
> > > >> >>> service some different provider.
> > > >> >>>
> > > >> >>> Maybe the rule should be to locate the operator according to the
> > > first
> > > >> >>> provider referenced.  So e.g. s3_to_gcs_transfer_operator.py
> would
> > > go
> > > >> to
> > > >> >>> aws.
> > > >> >>>
> > > >> >>>
> > > >> >>> On Tue, Jul 30, 2019 at 6:21 AM Kamil Breguła <
> > > >> kamil.bregula@polidea.com
> > > >> >>>
> > > >> >>> wrote:
> > > >> >>>
> > > >> >>>> Yes. All changes will be backwards compatible. In the case of
> > using
> > > >> >>>> the old path, a message containing a proposal for change will
> be
> > > >> >>>> reported to the user.
> > > >> >>>>
> > > >> >>>> I prepared an example of how to change the name of a class in a
> > > case
> > > >> >>>> with the use of a native solution.
> > > >> >>>> Source code:
> > > >> >>>>
> > > >> >>
> > > >>
> > >
> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/rename
> > > >> >>>> Preview from IDE: https://imgur.com/a/45NxS5W
> > > >> >>>>
> > > >> >>>> On Tue, Jul 30, 2019 at 2:28 PM Ash Berlin-Taylor <
> > ash@apache.org>
> > > >> >>> wrote:
> > > >> >>>>>
> > > >> >>>>> Just cos I'm not sure it's _explicitly_ stated, but all of the
> > > moves
> > > >> >>>> will have a deprecation of the old name right?
> > > >> >>>>>
> > > >> >>>>> 3+4 case D gets my vote too.
> > > >> >>>>>
> > > >> >>>>> -a
> > > >> >>>>>
> > > >> >>>>>> On 30 Jul 2019, at 09:58, Jarek Potiuk <
> > Jarek.Potiuk@polidea.com
> > > >
> > > >> >>>> wrote:
> > > >> >>>>>>
> > > >> >>>>>> I went ahead and updated the page (just to speed it up) as I
> > > think
> > > >> >> it
> > > >> >>>>>> really makes sense to join those two cases (and I do not see
> > any
> > > >> >>>> drawbacks
> > > >> >>>>>> - I think the options we have cover all possible approaches)
> > and
> > > we
> > > >> >>> can
> > > >> >>>>>> always go back if we need to.
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal
> > > >> >>>>>>
> > > >> >>>>>> My vote is *D*.
> > > >> >>>>>>
> > > >> >>>>>>  - airflow/contrib/operators/gcp_bigtable_operator.py  →
> > > >> >>> airflow/gcp
> > > >> >>>>>>  /operators/bigtable_operator.py
> > > >> >>>>>>  - airflow/contrib/operators/ssh_operator.py ->
> > > >> >>>>>>  airflow/operators/ssh_operator.py
> > > >> >>>>>>
> > > >> >>>>>> I updated the page with my vote.
> > > >> >>>>>> I propose that we vote till Friday on that one (all the rest
> is
> > > >> >>> already
> > > >> >>>>>> agreed).
> > > >> >>>>>>
> > > >> >>>>>> J.
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>> On Tue, Jul 30, 2019 at 9:42 AM Jarek Potiuk <
> > > >> >>> Jarek.Potiuk@polidea.com
> > > >> >>>>>
> > > >> >>>>>> wrote:
> > > >> >>>>>>
> > > >> >>>>>>> I think almost everyone voted and we have almost perfect
> > > >> >> consensus.
> > > >> >>>> We all
> > > >> >>>>>>> agree amongst other on moving all operators out of contrib
> > > >> >> (Great!).
> > > >> >>>>>>>
> > > >> >>>>>>> The only doubts are for *Case 3* (Cloud provider prefix) and
> > > *Case
> > > >> >>> 4*
> > > >> >>>>>>> (Using Namespaces).
> > > >> >>>>>>> I think there was actually an overlap between those two.
> Also
> > > >> >> Ash's
> > > >> >>>>>>> comment/veto on that is quite clear.
> > > >> >>>>>>> So I have final (I hope!) proposal to join those two cases
> and
> > > >> >> have
> > > >> >>>> one
> > > >> >>>>>>> voting (till Friday) only on that.
> > > >> >>>>>>>
> > > >> >>>>>>> I will update the doc if you all agree with me that makes
> more
> > > >> >> sense
> > > >> >>>> as
> > > >> >>>>>>> single case to vote on:
> > > >> >>>>>>>
> > > >> >>>>>>> *[Case 3 + Case 4]: Grouping Cloud Provider operators.*
> > > >> >>>>>>>
> > > >> >>>>>>> *Option A*. Keep operators/sensors/hooks in
> > > >> >>> airflow/operators(sensors,
> > > >> >>>>>>> hooks) and keep/add prefixes in file names.
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py  becomes
> > > >> >>>>>>>  airflow/operators/**aws_sns_publish_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> > > >> >>>>>>> becomes *airflow/operators/gcp_dataproc_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/hooks/gcp_bigtable_hook.py  *becomes
> > *airflow/*
> > > >> >>>>>>>  *hooks/gcp_bigtable_hook.py*
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes
> > *airflow/*
> > > >> >>>>>>>  *operators/ssh_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> *Option B*. Group operators/sensors/hooks in
> > > >> >>>> airflow/operators(sensors,
> > > >> >>>>>>> hooks)/*<PROVIDER>.* Non-cloud provider ones are moved to
> > > >> >>>>>>> airflow/operators(sensors/hooks), Drop the prefix.
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> > > >> >>>>>>>  becomes airflow/operators/**aws/sns_publish_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> > > >> >>>>>>> becomes *airflow/operators/gcp/dataproc_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py
> *becomes
> > > >> >>>> *airflow/*
> > > >> >>>>>>>  *operators/gcp/bigtable_operator.py*
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes
> > *airflow/*
> > > >> >>>>>>>  *operators/ssh_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> *Option C*. Group operators/sensors/hooks in
> > > >> >>>> airflow/operators(sensors,
> > > >> >>>>>>> hooks)*/<PROVIDER>.* Non-cloud provider ones are moved to
> > > >> >>>>>>> airflow/operators(sensors/hooks), Keep the prefix.
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> > > >> >>>>>>>  becomes
> airflow/operators/**aws/aws_sns_publish_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> > > >> >>>>>>> becomes *airflow/operators/gcp/gcp_dataproc_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py
> *becomes
> > > >> >>>> *airflow/*
> > > >> >>>>>>>  *operators/gcp/gcp_bigtable_operator.py*
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes
> > *airflow/*
> > > >> >>>>>>>  *operators/ssh_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> *Option D.* Group operators/sensors/hooks in
> > > >> >>>> *airflow/<PROVIDER>*/operators(sensors,
> > > >> >>>>>>> hooks). Non-cloud provider ones are moved to
> > > >> >>>>>>> airflow/operators(sensors/hooks). Drop the prefix.
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> > > >> >>>>>>>  becomes airflow/aws/operators/**sns_publish_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> > > >> >>>>>>> becomes *airflow/gcp/operators/dataproc_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py
> *becomes
> > > >> >>>> *airflow/*
> > > >> >>>>>>>  *gcp/operators/bigtable_operator.py*
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes
> > *airflow/*
> > > >> >>>>>>>  *operators/ssh_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> *Option E.* Group operators/sensors/hooks in
> > > >> >>>> *airflow/<PROVIDER>*/operators(sensors,
> > > >> >>>>>>> hooks). Non-cloud provider ones are moved to
> > > >> >>>>>>> airflow/operators(sensors/hooks).* Keep the prefix*.
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py
> > > >> >>>>>>>  becomes
> airflow/aws/operators/aws_**sns_publish_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  - *airflow/contrib/operators/dataproc_operator.py*
> > > >> >>>>>>> becomes *airflow/gcp/operators/gcp_dataproc_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py
> *becomes
> > > >> >>>> *airflow/*
> > > >> >>>>>>>  *gcp/operators/gcp_bigtable_operator.py*
> > > >> >>>>>>>  -
> > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py  *becomes
> > *airflow/*
> > > >> >>>>>>>  *operators/ssh_operator.py*
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> Shall I proceed with updating the page ?
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> J.
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> On Mon, Jul 29, 2019 at 11:18 AM Kaxil Naik <
> > > kaxilnaik@gmail.com>
> > > >> >>>> wrote:
> > > >> >>>>>>>
> > > >> >>>>>>>> Thanks Jarek, adding my vote.
> > > >> >>>>>>>>
> > > >> >>>>>>>> On Mon, Jul 29, 2019 at 2:40 PM Ash Berlin-Taylor <
> > > >> >> ash@apache.org>
> > > >> >>>> wrote:
> > > >> >>>>>>>>
> > > >> >>>>>>>>> Thanks Jarek, much clearer. I have voted there too.
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> -ash
> > > >> >>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>>> On 29 Jul 2019, at 08:13, Driesprong, Fokko
> > > >> >> <fokko@driesprong.frl
> > > >> >>>>
> > > >> >>>>>>>>> wrote:
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Thanks Jarek, the Wiki works much better for me. I've
> added
> > > my
> > > >> >>> vote
> > > >> >>>>>>>> there
> > > >> >>>>>>>>>> as well.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> You've convinced me on the second case. I've changed my
> > vote
> > > on
> > > >> >>>> Case 2
> > > >> >>>>>>>>> from
> > > >> >>>>>>>>>> A to B :-)
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Maybe we should leave the vote open a bit longer since it
> > > >> >>> involves
> > > >> >>>>>>>> quite
> > > >> >>>>>>>>>> some reading, and I would like to get some proper
> feedback
> > > and
> > > >> >>>>>>>> opinions
> > > >> >>>>>>>>> on
> > > >> >>>>>>>>>> this. Plus I only see two votes right now. If we don't
> > think
> > > >> >> this
> > > >> >>>>>>>>> through,
> > > >> >>>>>>>>>> it might be that we're having this vote again in the near
> > > >> >> future
> > > >> >>>> :-)
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Cheers, Fokko
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Op zo 28 jul. 2019 om 10:49 schreef Jarek Potiuk <
> > > >> >>>>>>>>> Jarek.Potiuk@polidea.com>:
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>>> I updated the AIP-21 proposal in the way that should
> > answer
> > > >> >> all
> > > >> >>>> the
> > > >> >>>>>>>>>>> concerns in this thread.
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> NOTE! There is an action for all of those who are
> > > interested:
> > > >> >>>> Please
> > > >> >>>>>>>>>>> fill-in your voting in the voting table till Tuesday
> 30th
> > of
> > > >> >>> July
> > > >> >>>> 6pm
> > > >> >>>>>>>>> CEST
> > > >> >>>>>>>>>>> (5pm BST, 9 am PST).
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> I created a summary of the proposal
> > > >> >>>>>>>>>>> <
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>> in table form - which contains also examples for each
> > case.
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> I added a voting table
> > > >> >>>>>>>>>>> <
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>> where you should add your votes:
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> I propose this voting mechanism:
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> Voting will take place till *Tuesday* 30 Jul 2019 6pm
> CEST
> > > (5
> > > >> >>> pm
> > > >> >>>>>>>> BST,
> > > >> >>>>>>>>> 9am
> > > >> >>>>>>>>>>> PST) .
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> After the choice, the final consistent set of choices
> will
> > > be
> > > >> >>>>>>>> announced
> > > >> >>>>>>>>>>> (taking into account majority of binding vote, also
> > > including
> > > >> >>>>>>>> potential
> > > >> >>>>>>>>>>> vetos and consistency between the choices. Non-binding
> > votes
> > > >> >>> will
> > > >> >>>> be
> > > >> >>>>>>>>> taken
> > > >> >>>>>>>>>>> into account in case there is a draw. The final set of
> > > choices
> > > >> >>>> will
> > > >> >>>>>>>> be
> > > >> >>>>>>>>>>> announced at devlist thread
> > > >> >>>>>>>>>>> <
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/4cedc94bee53ad908eee8333a56b58be8b5641881e73f69b97e436a9@%3Cdev.airflow.apache.org%3E
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>> after
> > > >> >>>>>>>>>>> the voting completes.
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> Unless there is a veto raised to the final proposal, the
> > > final
> > > >> >>>>>>>> proposal
> > > >> >>>>>>>>> is
> > > >> >>>>>>>>>>> accepted by Lazy Consensus
> > > >> >>>>>>>>>>> <
> > https://community.apache.org/committers/lazyConsensus.html
> > > >
> > > >> >>> on
> > > >> >>>>>>>>> *Friday*
> > > >> >>>>>>>>>>> 02
> > > >> >>>>>>>>>>> Aug 2019 at 6pm CEST (5 pm BST, 9am PST).
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> Please let me know if what I proposed can be further
> > > improved
> > > >> >> to
> > > >> >>>>>>>> avoid
> > > >> >>>>>>>>>>> chaos.
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> J.
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> On Sat, Jul 27, 2019 at 11:41 AM Jarek Potiuk <
> > > >> >>>>>>>> Jarek.Potiuk@polidea.com
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>>> Ok. I see that indeed voting could have been organised
> a
> > > bit
> > > >> >>>> better
> > > >> >>>>>>>> -
> > > >> >>>>>>>>> dev
> > > >> >>>>>>>>>>>> list does not seem to cope well with such a complex
> case
> > > :).
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> As Kamil noticed - we already have a formal AIP-21
> > > >> >>>>>>>>>>>> <
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> in the Wiki - which I should have mentioned before. I
> > have
> > > >> >> not
> > > >> >>>> much
> > > >> >>>>>>>>> time
> > > >> >>>>>>>>>>>> today - but tomorrow I will try to summarise the
> options
> > -
> > > >> >>> based
> > > >> >>>> on
> > > >> >>>>>>>>> some
> > > >> >>>>>>>>>>>> real code examples (to make it easier to digest). I
> think
> > > the
> > > >> >>>> best
> > > >> >>>>>>>>>>> approach
> > > >> >>>>>>>>>>>> will be to make a voting matrix in the AIP-21 Wiki page
> > > where
> > > >> >>>> people
> > > >> >>>>>>>>> will
> > > >> >>>>>>>>>>>> be able to state their preferences for each case (by
> > > adding a
> > > >> >>>> row to
> > > >> >>>>>>>>> the
> > > >> >>>>>>>>>>>> table). I think that might provide much better clarity
> > and
> > > a
> > > >> >>>> single
> > > >> >>>>>>>>> place
> > > >> >>>>>>>>>>>> which will show the current aggregated state of voting.
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> However - as Ash also stated - some of the options are
> > > >> >>>>>>>> inter-dependent
> > > >> >>>>>>>>> so
> > > >> >>>>>>>>>>>> at the end we will have to adjust the results in case
> > they
> > > >> >> are
> > > >> >>>> not
> > > >> >>>>>>>>>>>> consistent  - and make final voting on aggregated set
> of
> > > >> >> cases
> > > >> >>> -
> > > >> >>>>>>>> but I
> > > >> >>>>>>>>>>>> think in this case following "lasy consensus" - i,e. if
> > > noone
> > > >> >>>>>>>> objects
> > > >> >>>>>>>>> to
> > > >> >>>>>>>>>>>> final set of cases, we will proceed with it..
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> Do you think that will work better ?
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> J.
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> Principal Software Engineer
> > > >> >>>>>>>>>>>> Phone: +48660796129
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>> sob., 27 lip 2019, 09:26 użytkownik Kamil Breguła <
> > > >> >>>>>>>>>>>> kamil.bregula@polidea.com> napisał:
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>>> Document is also available as AIP on wiki:
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>> On Sat, Jul 27, 2019, 9:07 AM Driesprong, Fokko
> > > >> >>>>>>>> <fokko@driesprong.frl
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>> I agree with all, this is a bit chaotic. For the sake
> > of
> > > >> >>>> clarity,
> > > >> >>>>>>>> I
> > > >> >>>>>>>>>>>>> would
> > > >> >>>>>>>>>>>>>> suggest moving the Google Doc to the Wiki as an AIP.
> > > >> >> Create a
> > > >> >>>>>>>>>>> structural
> > > >> >>>>>>>>>>>>>> code improvement AIP and then and add a matrix of
> > > >> >> users/cases
> > > >> >>>>>>>> where
> > > >> >>>>>>>>>>>>>> everybody can add his vote per case. This gives also
> > the
> > > >> >>>>>>>> opportunity
> > > >> >>>>>>>>>>> for
> > > >> >>>>>>>>>>>>>> changing your vote on a specific case. WDYT?
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>> Cheers, Fokko
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>> Op vr 26 jul. 2019 om 22:28 schreef Chen Tong <
> > > >> >>>> cixuuz@gmail.com>:
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>> I agree with Ash. It is clear to vote on each case
> > > rather
> > > >> >>> than
> > > >> >>>>>>>> all
> > > >> >>>>>>>>>>>>>>> together.
> > > >> >>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:54 PM Ash Berlin-Taylor <
> > > >> >>>>>>>> ash@apache.org>
> > > >> >>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>> A number of proposals overlap or add on to each
> > other,
> > > >> >> so I
> > > >> >>>>>>>> think
> > > >> >>>>>>>>>>>>> (?) a
> > > >> >>>>>>>>>>>>>>>> single vote makes sense.
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>> To be clear, my vote is -1 until it's clear exactly
> > > what
> > > >> >> we
> > > >> >>>> are
> > > >> >>>>>>>>>>>>> voting
> > > >> >>>>>>>>>>>>>> on.
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>> On 26 July 2019 20:39:56 BST, Kaxil Naik <
> > > >> >>>> kaxilnaik@gmail.com>
> > > >> >>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>> I agree with Ash and instead of voting on all 7
> > Cases
> > > >> >>>> together,
> > > >> >>>>>>>>>>> how
> > > >> >>>>>>>>>>>>>>>>> about
> > > >> >>>>>>>>>>>>>>>>> separate vote emails for all the cases? Each email
> > can
> > > >> >>> have
> > > >> >>>> the
> > > >> >>>>>>>>>>>>>>>>> description
> > > >> >>>>>>>>>>>>>>>>> copied over from the doc.
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>> It would probably be a bit easier to track a
> > decision
> > > in
> > > >> >>> the
> > > >> >>>>>>>>>>> future
> > > >> >>>>>>>>>>>>> as
> > > >> >>>>>>>>>>>>>>>>> well.
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>> What do you guys think? @Jarek Potiuk <
> > > >> >>>>>>>> Jarek.Potiuk@polidea.com>
> > > >> >>>>>>>>>>>>>>>>> @Driesprong,
> > > >> >>>>>>>>>>>>>>>>> Fokko <fokko@driesprong.frl>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>> On Sat, Jul 27, 2019 at 1:03 AM Kaxil Naik <
> > > >> >>>>>>>> kaxilnaik@gmail.com>
> > > >> >>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> *Case #1* airflow.contrib.{resources} : *Solution
> > A *
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> *Case #2* git *_{operator/sensor}{/s}.py :
> > *Solution
> > > A
> > > >> >> *
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> *Case #3* {aws/azure/gcp}_* : *Solution C*
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> *Case #4* Separate namespace for resources :
> > > >> >>>>>>>>>>>>>>>>>> *Solution A*
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> *Case #5* *Operator : *Solution A*
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> *Case #7* : *Solution A*
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 11:09 PM Daniel Standish
> > > >> >>>>>>>>>>>>>>>>> <dpstandish@gmail.com>
> > > >> >>>>>>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> I have tried to add some clarification to
> Jarek's
> > > >> >>> summary,
> > > >> >>>>>>>> but
> > > >> >>>>>>>>>>> I
> > > >> >>>>>>>>>>>>> am
> > > >> >>>>>>>>>>>>>>>>> a
> > > >> >>>>>>>>>>>>>>>>>>> little fuzzy on exact proposal for case 3 so
> > > hopefully
> > > >> >>>> Jarek
> > > >> >>>>>>>>>>> can
> > > >> >>>>>>>>>>>>>>>>> further
> > > >> >>>>>>>>>>>>>>>>>>> clarify.
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> According to my reading, in this motion cases
> > 4,5,6
> > > >> >> are
> > > >> >>>> all
> > > >> >>>>>>>>>>>>> either
> > > >> >>>>>>>>>>>>>>>>>>> proposal
> > > >> >>>>>>>>>>>>>>>>>>> *rejections* or otherwise have no effect, so
> > > attention
> > > >> >>>> can be
> > > >> >>>>>>>>>>>>>>>>> focused on
> > > >> >>>>>>>>>>>>>>>>>>> cases 1,2,3 and 7.
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> *Motion is to adopt the following:*
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 1 - Solution A - Get rid of contrib
> > > >> >>>>>>>>>>>>>>>>>>> All objects will be moved out of contrib folder
> > and
> > > >> >> into
> > > >> >>>>>>>>>>>>> respective
> > > >> >>>>>>>>>>>>>>>>>>> non-contrib folder.
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 2 - Solution A - Remove object type suffix
> > from
> > > >> >>>> modules
> > > >> >>>>>>>>>>>>>>>>>>> Example:
> > > >> >>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator
> becomes
> > > >> >>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator
> > > >> >>>>>>>>>>>>>>>>>>> Example:
> > > >> >>>>>>>>>>>>>>>>>>> Airflow.hooks.foo_hook.FooOperator becomes
> > > >> >>>>>>>>>>>>> airflow.hooks.foo.FooHook
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 3 - conventions for cloud provider etc
> > prefixes
> > > >> >>>>>>>>>>>>>>>>>>> *I am not sure exactly what the proposal is.*
> > > >> >>>>>>>>>>>>>>>>>>> Example case is
> > > >> >>>>>>>>>>>>> airflow/contrib/operators/gcp_bigtable_operator.py
> > > >> >>>>>>>>>>>>>>>>>>> Since motion adopts case 1 solution A, we can
> > assume
> > > >> >> it
> > > >> >>> is
> > > >> >>>>>>>> now
> > > >> >>>>>>>>>>>>> named
> > > >> >>>>>>>>>>>>>>>>> like
> > > >> >>>>>>>>>>>>>>>>>>> so: airflow/operators/gcp_bigtable_operator.py
> > > >> >>>>>>>>>>>>>>>>>>> So what is proposed for this case?
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 4 - Solution B - *no change*
> > > >> >>>>>>>>>>>>>>>>>>> This proposal was about namespaces but the
> motion
> > is
> > > >> >> to
> > > >> >>>>>>>> reject
> > > >> >>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>> proposal.
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 5 - Solution B - *no change*
> > > >> >>>>>>>>>>>>>>>>>>> The proposal was to remove class type suffix
> e.g.
> > > >> >> remove
> > > >> >>>> the
> > > >> >>>>>>>>>>>>>>>>> Operator in
> > > >> >>>>>>>>>>>>>>>>>>> FooOperator, but the motion is to reject this
> > > proposal
> > > >> >>> --
> > > >> >>>>>>>> i.e.
> > > >> >>>>>>>>>>>>>>>>> motion is
> > > >> >>>>>>>>>>>>>>>>>>> no
> > > >> >>>>>>>>>>>>>>>>>>> change on this case, i.e. we resolve to keep
> > > >> >> "Operator"
> > > >> >>>> (and
> > > >> >>>>>>>>>>>>>>>>> "Sensor")
> > > >> >>>>>>>>>>>>>>>>>>> suffixes on class names
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 6 - *No change*
> > > >> >>>>>>>>>>>>>>>>>>> This case concerns object naming
> inconsistencies,
> > > but
> > > >> >>>> motion
> > > >> >>>>>>>>>>> does
> > > >> >>>>>>>>>>>>>>>>> not
> > > >> >>>>>>>>>>>>>>>>>>> enact
> > > >> >>>>>>>>>>>>>>>>>>> any specific proposal.
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Case 7 - Solution A - use warnings.warn template
> > for
> > > >> >>>>>>>>>>> deprecation
> > > >> >>>>>>>>>>>>>>>>> warnings
> > > >> >>>>>>>>>>>>>>>>>>> (instead of zope)
> > > >> >>>>>>>>>>>>>>>>>>> Example:
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> # in old_package/old_mod.py
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> import warnings
> > > >> >>>>>>>>>>>>>>>>>>> from new_package.new_mod import *
> > > >> >>>>>>>>>>>>>>>>>>> warnings.warn("old_package.old_mod has moved to
> > > >> >>>>>>>>>>>>> new_package.new_mod.
> > > >> >>>>>>>>>>>>>>>>>>> Import
> > > >> >>>>>>>>>>>>>>>>>>> of "
> > > >> >>>>>>>>>>>>>>>>>>> "old_package.old_mod will become unsupported in
> > > >> >> version
> > > >> >>>> 2",
> > > >> >>>>>>>>>>>>>>>>>>> DeprecationWarning, 2)
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> Reference implementation here:
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/solution1
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> FWIW +1 (non-binding) on 1,2,7 -- unsure on 3.
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> I am very happy that this motion now gets rid of
> > > >> >>> contrib.
> > > >> >>>>>>>>>>> Thank
> > > >> >>>>>>>>>>>>> you
> > > >> >>>>>>>>>>>>>>>>>>> Sergio
> > > >> >>>>>>>>>>>>>>>>>>> for turning the tide with your effective
> > > argumentation
> > > >> >>> ;)
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:16 AM Ash
> Berlin-Taylor
> > <
> > > >> >>>>>>>>>>>>> ash@apache.org>
> > > >> >>>>>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>> Could someone summarise what the proposed name
> > > >> >> changes
> > > >> >>>> are,
> > > >> >>>>>>>>>>>>> here,
> > > >> >>>>>>>>>>>>>>>>> in
> > > >> >>>>>>>>>>>>>>>>>>> this
> > > >> >>>>>>>>>>>>>>>>>>>> thread? Pointing at a google doc and only
> giving
> > > >> >>> partial
> > > >> >>>>>>>>>>>>> examples
> > > >> >>>>>>>>>>>>>>>>> is 1)
> > > >> >>>>>>>>>>>>>>>>>>> a
> > > >> >>>>>>>>>>>>>>>>>>>> bit difficult to quickly work out what we are
> > > voting
> > > >> >>> on,
> > > >> >>>> and
> > > >> >>>>>>>>>>> 2)
> > > >> >>>>>>>>>>>>>>>>> using a
> > > >> >>>>>>>>>>>>>>>>>>>> google doc isn't great for a longer term
> record.
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>> Thanks,
> > > >> >>>>>>>>>>>>>>>>>>>> Ash
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> On 26 Jul 2019, at 09:14, Jarek Potiuk
> > > >> >>>>>>>>>>>>>>>>> <Jarek.Potiuk@polidea.com>
> > > >> >>>>>>>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> I would like to recast the vote. Let's start
> > from
> > > 0
> > > >> >> :)
> > > >> >>>> . I
> > > >> >>>>>>>>>>>>> would
> > > >> >>>>>>>>>>>>>>>>> like
> > > >> >>>>>>>>>>>>>>>>>>> to
> > > >> >>>>>>>>>>>>>>>>>>>>> keep the July 30th 6pm CEST date, and at least
> > > three
> > > >> >>> +1
> > > >> >>>>>>>>>>>>>>>>> (binding)
> > > >> >>>>>>>>>>>>>>>>>>> votes
> > > >> >>>>>>>>>>>>>>>>>>>> are
> > > >> >>>>>>>>>>>>>>>>>>>>> needed to pass. I marked in red the
> > modifications
> > > to
> > > >> >>> the
> > > >> >>>>>>>>>>>>>>>>> previous
> > > >> >>>>>>>>>>>>>>>>>>>> proposal.
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> Here is the proposal (details here
> > > >> >>>>>>>>>>>>>>>>>>>>> <
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#heading=h.j4jc3i70qszo
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> )
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 1: A: Get rid of contrib,*
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 2: A:
> > > >> >>> Airflow.operators.foo_operator.FooOperator
> > > >> >>>>>>>>>>>>> could
> > > >> >>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator*
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 3: C:
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>
> > > airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator
> > > >> >>>>>>>>>>>>>>>>>>> could
> > > >> >>>>>>>>>>>>>>>>>>>>> become
> > > >> >>> airflow.gcp.operators.bigtable.BigTableOperator*
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction*
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor")
> > > >> >> suffixes
> > > >> >>> on
> > > >> >>>>>>>>>>>>> class
> > > >> >>>>>>>>>>>>>>>>>>> names*
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on
> > > >> >>> case-by-case
> > > >> >>>>>>>>>>>>> (and
> > > >> >>>>>>>>>>>>>>>>> my team
> > > >> >>>>>>>>>>>>>>>>>>>> can
> > > >> >>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)*
> > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 7: Native python solution (with better
> > IDE
> > > >> >>>>>>>>>>>>>>>>> integration)*
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> This is my binding (+1) vote.
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 10:08 AM Jarek Potiuk
> <
> > > >> >>>>>>>>>>>>>>>>>>> Jarek.Potiuk@polidea.com>
> > > >> >>>>>>>>>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> Hey Felix,
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> For 7* -> I am in favour of trying the native
> > one
> > > >> >> as
> > > >> >>>> well
> > > >> >>>>>>>>>>> (I
> > > >> >>>>>>>>>>>>>>>>> use IDE
> > > >> >>>>>>>>>>>>>>>>>>>> quite
> > > >> >>>>>>>>>>>>>>>>>>>>>> often ;) )
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> J.
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 24, 2019 at 9:34 AM Driesprong,
> > Fokko
> > > >> >>>>>>>>>>>>>>>>>>> <fokko@driesprong.frl
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> wrote:
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Thanks Kamil for putting the document
> > together.
> > > I
> > > >> >>>> wasn't
> > > >> >>>>>>>>>>>>> able
> > > >> >>>>>>>>>>>>>>>>> to
> > > >> >>>>>>>>>>>>>>>>>>>> respond
> > > >> >>>>>>>>>>>>>>>>>>>>>>> earlier since I was giving Airflow workshops
> > > >> >>>> throughout
> > > >> >>>>>>>>>>>>> Europe
> > > >> >>>>>>>>>>>>>>>>> :-)
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 1: *Solution A → Remove everything
> from
> > > >> >>> contrib
> > > >> >>>>>>>>>>> into
> > > >> >>>>>>>>>>>>> a
> > > >> >>>>>>>>>>>>>>>>> single
> > > >> >>>>>>>>>>>>>>>>>>>>>>> package.
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> To make it easier to the end-user, my
> > preference
> > > >> >>>> would be
> > > >> >>>>>>>>>>>>> to
> > > >> >>>>>>>>>>>>>>>>> get
> > > >> >>>>>>>>>>>>>>>>>>> rid of
> > > >> >>>>>>>>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>>>>>> contrib package altogether. What's in
> contrib
> > or
> > > >> >> not
> > > >> >>>> is a
> > > >> >>>>>>>>>>>>>>>>> tricky
> > > >> >>>>>>>>>>>>>>>>>>>> question,
> > > >> >>>>>>>>>>>>>>>>>>>>>>> I think this is already reflected in the
> > > document
> > > >> >>>> since
> > > >> >>>>>>>>>>>>>>>>> "maturity"
> > > >> >>>>>>>>>>>>>>>>>>> is
> > > >> >>>>>>>>>>>>>>>>>>>> in
> > > >> >>>>>>>>>>>>>>>>>>>>>>> quotes. What would be the moment to transfer
> > the
> > > >> >>>> operator
> > > >> >>>>>>>>>>>>> to
> > > >> >>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>> core
> > > >> >>>>>>>>>>>>>>>>>>>> or
> > > >> >>>>>>>>>>>>>>>>>>>>>>> vice versa? I'm afraid that renaming contrib
> > to
> > > >> >>>>>>>>>>> incubating
> > > >> >>>>>>>>>>>>>>>>> will
> > > >> >>>>>>>>>>>>>>>>>>> confuse
> > > >> >>>>>>>>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>>>>>> user even more. For people who aren't as
> known
> > > >> >> with
> > > >> >>>>>>>>>>>>> Airflow it
> > > >> >>>>>>>>>>>>>>>>> isn't
> > > >> >>>>>>>>>>>>>>>>>>>> that
> > > >> >>>>>>>>>>>>>>>>>>>>>>> transparent which operator lives where,
> > > especially
> > > >> >>> if
> > > >> >>>> you
> > > >> >>>>>>>>>>>>>>>>> don't use
> > > >> >>>>>>>>>>>>>>>>>>> an
> > > >> >>>>>>>>>>>>>>>>>>>> IDE
> > > >> >>>>>>>>>>>>>>>>>>>>>>> such as Ash ;) Besides that I don't think it
> > is
> > > >> >>> worth
> > > >> >>>>>>>>>>>>> changing
> > > >> >>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>>> public
> > > >> >>>>>>>>>>>>>>>>>>>>>>> API just for a different name.
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> Ok. I am convinced. I will re-cast the vote
> > with
> > > >> >>> this.
> > > >> >>>> It
> > > >> >>>>>>>>>>>>> will
> > > >> >>>>>>>>>>>>>>>>> make
> > > >> >>>>>>>>>>>>>>>>>>>> easier
> > > >> >>>>>>>>>>>>>>>>>>>>>> as we will not have to define other rules as
> > > those
> > > >> >>>> that we
> > > >> >>>>>>>>>>>>>>>>> already
> > > >> >>>>>>>>>>>>>>>>>>> have.
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 2:* Slight preference to Solution B
> > (let's
> > > >> >>> say a
> > > >> >>>>>>>>>>> +0)
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> I like to search on Github with t, and then
> > > search
> > > >> >>> for
> > > >> >>>>>>>>>>>>>>>>> BashOperator.
> > > >> >>>>>>>>>>>>>>>>>>>> After
> > > >> >>>>>>>>>>>>>>>>>>>>>>> the change, you should search for
> > Operator/Bash.
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Jarek, I'm confused with your vote on this,
> > from
> > > >> >>> your
> > > >> >>>>>>>>>>> mail
> > > >> >>>>>>>>>>>>> you
> > > >> >>>>>>>>>>>>>>>>>>> state: -
> > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 2: B:
> > > >> >>> Airflow.operators.foo_operator.FooOperator
> > > >> >>>>>>>>>>>>> could
> > > >> >>>>>>>>>>>>>>>>> become
> > > >> >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator*. But the
> B
> > > >> >>>> solution is
> > > >> >>>>>>>>>>>>>>>>> keeping it
> > > >> >>>>>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>>>>>> same, so that would mean - *Case 2: B:
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator
> > > *would
> > > >> >>> stay
> > > >> >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo_operator*.FooOperator*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> My mistake. It was of course A. I think
> there
> > > is a
> > > >> >>>> little
> > > >> >>>>>>>>>>>>>>>>> confusion
> > > >> >>>>>>>>>>>>>>>>>>>> here.
> > > >> >>>>>>>>>>>>>>>>>>>>>> This case is not for changing BashOperator
> into
> > > >> >> Bash
> > > >> >>>> but
> > > >> >>>>>>>>>>> for
> > > >> >>>>>>>>>>>>>>>>> changing
> > > >> >>>>>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>>>>> *module* name not the *class* name. So
> > > >> >>>>>>>>>>>>>>>>> bash_operator.BashOperator ->
> > > >> >>>>>>>>>>>>>>>>>>>>>> bash.BashOperator :) . you will still search
> > for
> > > >> >>>>>>>>>>>>> BashOperator
> > > >> >>>>>>>>>>>>>>>>> in this
> > > >> >>>>>>>>>>>>>>>>>>>> case.
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 3:* I'm leaning towards B
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Mostly because it isn't as trivial to decide
> > > when
> > > >> >> it
> > > >> >>>> gets
> > > >> >>>>>>>>>>>>> its
> > > >> >>>>>>>>>>>>>>>>> own
> > > >> >>>>>>>>>>>>>>>>>>>> package
> > > >> >>>>>>>>>>>>>>>>>>>>>>> or not. Besides the cloud providers, there
> are
> > > >> >>>> companies
> > > >> >>>>>>>>>>>>> like
> > > >> >>>>>>>>>>>>>>>>>>>> Databricks
> > > >> >>>>>>>>>>>>>>>>>>>>>>> and Qubole which might also end up with
> their
> > > own
> > > >> >>>> package
> > > >> >>>>>>>>>>>>>>>>> because
> > > >> >>>>>>>>>>>>>>>>>>> their
> > > >> >>>>>>>>>>>>>>>>>>>>>>> integration is getting richer over time.
> > Moving
> > > >> >> this
> > > >> >>>> is a
> > > >> >>>>>>>>>>>>> bit
> > > >> >>>>>>>>>>>>>>>>>>> painful
> > > >> >>>>>>>>>>>>>>>>>>>>>>> because we need to deprecate the old path
> and
> > > move
> > > >> >>>>>>>>>>>>> everything
> > > >> >>>>>>>>>>>>>>>>> which
> > > >> >>>>>>>>>>>>>>>>>>>>>>> introduces noise into version control etc.
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> I'd say, we should go for C. As I proposed.
> > It's
> > > >> >> not
> > > >> >>> as
> > > >> >>>>>>>>>>>>>>>>> difficult as
> > > >> >>>>>>>>>>>>>>>>>>> it
> > > >> >>>>>>>>>>>>>>>>>>>>>> seems to group operators in packages and it
> has
> > > >> >>>> numerous
> > > >> >>>>>>>>>>>>>>>>> advantages.
> > > >> >>>>>>>>>>>>>>>>>>> The
> > > >> >>>>>>>>>>>>>>>>>>>>>> nice thing about deprecations - we can do
> them
> > in
> > > >> >> the
> > > >> >>>>>>>>>>>>>>>>> __init__.py of
> > > >> >>>>>>>>>>>>>>>>>>> the
> > > >> >>>>>>>>>>>>>>>>>>>>>> packages which causes the noise fairly small
> > (Git
> > > >> >>> will
> > > >> >>>>>>>>>>>>> actually
> > > >> >>>>>>>>>>>>>>>>>>> realise
> > > >> >>>>>>>>>>>>>>>>>>>>>> that the file has been moved and you can
> follow
> > > >> >> that.
> > > >> >>>>>>>>>>> Maybe
> > > >> >>>>>>>>>>>>> not
> > > >> >>>>>>>>>>>>>>>>> as
> > > >> >>>>>>>>>>>>>>>>>>> super
> > > >> >>>>>>>>>>>>>>>>>>>>>> easy to merge changes later to 1.10.4  but
> it's
> > > >> >> not a
> > > >> >>>>>>>>>>> rocket
> > > >> >>>>>>>>>>>>>>>>> science
> > > >> >>>>>>>>>>>>>>>>>>>> either.
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 7:* Python native solution
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> Yes.
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> ---
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Apart from all, great work!
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Cheers, Fokko
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>> Op di 23 jul. 2019 om 21:27 schreef Felix
> > > >> >> Uellendall
> > > >> >>>>>>>>>>>>>>>>>>>>>>> <feluelle@pm.me.invalid
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> :
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> I have no strong "No" against any proposed
> > > change
> > > >> >>> of
> > > >> >>>>>>>>>>> these
> > > >> >>>>>>>>>>>>>>>>> cases.
> > > >> >>>>>>>>>>>>>>>>>>> So I
> > > >> >>>>>>>>>>>>>>>>>>>>>>> go
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> with +1 (non-binding).
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> P.S. Thanks Jarek for bringing this up
> again
> > > and
> > > >> >>> your
> > > >> >>>>>>>>>>>>> intense
> > > >> >>>>>>>>>>>>>>>>> work
> > > >> >>>>>>>>>>>>>>>>>>>>>>> towards
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> airflow currently :) and thanks to Kamil
> for
> > > even
> > > >> >>>>>>>>>>> creating
> > > >> >>>>>>>>>>>>>>>>> this
> > > >> >>>>>>>>>>>>>>>>>>>>>>> document. I
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> like how the code is getting more and more
> > > >> >>> consistent
> > > >> >>>>>>>>>>> and
> > > >> >>>>>>>>>>>>>>>>> clean :)
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> Kind regards,
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> Felix
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> Sent from ProtonMail mobile
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> -------- Original Message --------
> > > >> >>>>>>>>>>>>>>>>>>>>>>>> On Jul 23, 2019, 18:34, Jarek Potiuk wrote:
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone,
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> This email is calling a vote on the
> changes
> > in
> > > >> >>>> import
> > > >> >>>>>>>>>>>>> paths.
> > > >> >>>>>>>>>>>>>>>>> It's
> > > >> >>>>>>>>>>>>>>>>>>>> been
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> discussed in
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/4e648d9421c792d4537f5ac66f1a16dce468f816fc5221a9f9db9433@%3Cdev.airflow.apache.org%3E
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> The vote will last for at least 1 week
> (July
> > > >> >> 30th
> > > >> >>>> 6pm
> > > >> >>>>>>>>>>>>> CEST),
> > > >> >>>>>>>>>>>>>>>>> and
> > > >> >>>>>>>>>>>>>>>>>>> at
> > > >> >>>>>>>>>>>>>>>>>>>>>>> least
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> three +1 (binding) votes have been cast.
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> The proposal to vote is based on the
> > document
> > > >> >> from
> > > >> >>>>>>>>>>> Kamil
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> <
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> The proposed solution is:
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 1: B: Contrib vs not: we move all
> > that
> > > >> >> are
> > > >> >>>>>>>>>>> "well"
> > > >> >>>>>>>>>>>>>>>>> tested
> > > >> >>>>>>>>>>>>>>>>>>> and
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> rename contrib to "incubating" or
> similar.*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 2: B:
> > > >> >>>>>>>>>>> Airflow.operators.foo_operator.FooOperator
> > > >> >>>>>>>>>>>>>>>>> could
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 3: C:
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>
> > > airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator
> > > >> >>>>>>>>>>>>>>>>>>>> could
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> become
> > > >> >>>> airflow.gcp.operators.bigtable.BigTableOperator*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and
> "Sensor")
> > > >> >>>> suffixes
> > > >> >>>>>>>>>>> on
> > > >> >>>>>>>>>>>>>>>>> class
> > > >> >>>>>>>>>>>>>>>>>>> names*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on
> > > >> >>>> case-by-case
> > > >> >>>>>>>>>>>>> (and
> > > >> >>>>>>>>>>>>>>>>> my
> > > >> >>>>>>>>>>>>>>>>>>> team
> > > >> >>>>>>>>>>>>>>>>>>>>>>> can
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)*
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> This is my (binding) +1 vote.
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> J.
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> --
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> |
> > > Principal
> > > >> >>>>>>>>>>> Software
> > > >> >>>>>>>>>>>>>>>>> Engineer
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> M: [+48 660 796 129](tel:+48660796129)
> > > >> >>>>>>>>>>>>>>>>>>>>>>> <[+48660796129](tel:+48660796129)>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <
> https://www.polidea.com/>
> > > >> >>>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> --
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk
> > > >> >>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> |
> Principal
> > > >> >>>> Software
> > > >> >>>>>>>>>>>>>>>>> Engineer
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129>
> > > >> >>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> --
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> Jarek Potiuk
> > > >> >>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> |
> Principal
> > > >> >>> Software
> > > >> >>>>>>>>>>>>>> Engineer
> > > >> >>>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129>
> > > >> >>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>> --
> > > >> >>>>>>>>>>>>>>>>>> *Kaxil Naik*
> > > >> >>>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer*
> > > >> >>>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer |
> > *Certified*
> > > >> >>> Apache
> > > >> >>>>>>>>>>> Spark
> > > >> >>>>>>>>>>>>> &
> > > >> >>>>>>>>>>>>>>>>> Neo4j
> > > >> >>>>>>>>>>>>>>>>>> Developer
> > > >> >>>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> > > >> >>>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>>> --
> > > >> >>>>>>>>>>>>>>>>> *Kaxil Naik*
> > > >> >>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer*
> > > >> >>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer |
> *Certified*
> > > >> >>> Apache
> > > >> >>>>>>>> Spark
> > > >> >>>>>>>>>>> &
> > > >> >>>>>>>>>>>>>>>>> Neo4j
> > > >> >>>>>>>>>>>>>>>>> Developer
> > > >> >>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> > > >> >>>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>>
> > > >> >>>>>>>>>>>>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> --
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> Jarek Potiuk
> > > >> >>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software
> > > >> >>> Engineer
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>>> M: +48 660 796 129 <+48660796129>
> > > >> >>>>>>>>>>> [image: Polidea] <https://www.polidea.com/>
> > > >> >>>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>>>>> --
> > > >> >>>>>>>> *Kaxil Naik*
> > > >> >>>>>>>> *Big Data Consultant | DevOps Data Engineer*
> > > >> >>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache
> > > >> >> Spark &
> > > >> >>>> Neo4j
> > > >> >>>>>>>> Developer
> > > >> >>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil
> > > >> >>>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> --
> > > >> >>>>>>>
> > > >> >>>>>>> Jarek Potiuk
> > > >> >>>>>>> Polidea <https://www.polidea.com/> | Principal Software
> > > Engineer
> > > >> >>>>>>>
> > > >> >>>>>>> M: +48 660 796 129 <+48660796129>
> > > >> >>>>>>> [image: Polidea] <https://www.polidea.com/>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>
> > > >> >>>>>> --
> > > >> >>>>>>
> > > >> >>>>>> Jarek Potiuk
> > > >> >>>>>> Polidea <https://www.polidea.com/> | Principal Software
> > Engineer
> > > >> >>>>>>
> > > >> >>>>>> M: +48 660 796 129 <+48660796129>
> > > >> >>>>>> [image: Polidea] <https://www.polidea.com/>
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > > >>
> > >
> > > --
> > >
> > > Jarek Potiuk
> > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >
> > > M: +48 660 796 129 <+48660796129>
> > > [image: Polidea] <https://www.polidea.com/>
> > >
> >
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message