airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Leslie-Waksman <geo...@cloverhealth.com.INVALID>
Subject Re: What are the advantages of plugins, not sure I see any?
Date Thu, 29 Mar 2018 23:56:28 GMT
It's presumably useful if you want to package your plugins for other people
to use but it seems like everyone just adds those directly to the Airflow
codebase these days.

On Thu, Mar 29, 2018 at 4:27 PM Kyle Hamlin <hamlin.kn@gmail.com> wrote:

> Yeah so far I have only written hooks and operators so maybe the benefit
> only  kicks in for other airflow abstractions.
>
> > On Mar 29, 2018, at 7:15 PM, George Leslie-Waksman <
> george@cloverhealth.com.INVALID> wrote:
> >
> > We also import our operators and sensors directly.
> >
> > However, executors and some other pieces are a little bit harder to deal
> > with as non-plugins
> >
> >> On Thu, Mar 29, 2018 at 3:56 PM Kyle Hamlin <hamlin.kn@gmail.com>
> wrote:
> >>
> >> Hello,
> >>
> >> I just got done writing a few plugins, and the process has left me
> >> wondering what the real benefits are? As far as I can tell, it makes
> >> testing more difficult since you cannot import from the created module,
> you
> >> have to import directly from the plugin. Additionally, your code editor
> >> isn't aware of these new plugin modules since they are created when you
> >> start the app up, this makes it seem like there are errors when there
> >> aren't. Why not just create a lib/ dir with hooks, operators etc.. dirs
> >> inside and be done with it? Very curious what peoples thoughts are, who
> >> knows I could be testing wrong or writing the plugins wrong. Thanks in
> >> advance!
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message