airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamil BreguĊ‚a <kamil.breg...@polidea.com>
Subject Re: Manipulating the DAG Code View?
Date Tue, 13 Aug 2019 22:18:56 GMT
It seems to me that a better idea would be to add documentation to DAG.
https://airflow.readthedocs.io/en/latest/concepts.html?highlight=doc_md#documentation-notes

On Tue, Aug 13, 2019 at 8:55 PM Shaw, Damian P.
<damian.shaw.2@credit-suisse.com> wrote:
>
> As Ash says, I tried creating some test DAGs in 1.10.x and found that they didn't run.
There wasn't even any obvious error, Airflow loaded the file (I could see pyc created) but
it never appeared in the DAG bag.
>
> Damian
>
> -----Original Message-----
> From: Ash Berlin-Taylor [mailto:ash@apache.org]
> Sent: Tuesday, August 13, 2019 2:47 PM
> To: dev@airflow.apache.org; Chris Palmer
> Subject: Re: Manipulating the DAG Code View?
>
> Be carefully doing that in 1.10.4 - if you change that you may find your dags don't run
anymore!
>
> On 13 August 2019 19:46:01 BST, Chris Palmer <chris@crpalmer.com> wrote:
> >A more involved PR like you suggest might be valuable in the long run,
> >but
> >in the short term I've been successful in the past with simply
> >modifying
> >the fileloc attribute of DAGs. It get's set here
> ><https://github.com/apache/airflow/blob/master/airflow/models/dag.py#L229>
> >to the previous frame, but you could override it once you've created
> >the
> >DAG object.
> >
> >Chris
> >
> >On Tue, Aug 13, 2019 at 2:31 PM Shaw, Damian P. <
> >damian.shaw.2@credit-suisse.com> wrote:
> >
> >> Hi all,
> >>
> >> I'm in need of being able to manipulate the code view for many of my
> >DAGs
> >> which are generated by a single Python script that reads config
> >files. My
> >> DAG creation script is getting increasingly less useful to the end
> >users so
> >> I would like to instead present the config file that was used to
> >generate
> >> the DAG.
> >>
> >> I took a look at the relevant code vide code in Airflow:
> >>
> >https://github.com/apache/airflow/blob/3e2a02751cf890b780bc26b40c7cee7f1f4e0bd9/airflow/www/views.py#L433
> >>
> >> It seems it would be a relatively easy change to add an extra option
> >on
> >> the DAG, e.g. "dag.code_display_file" and maybe also a Lexer option,
> >then
> >> detect if this option has been set for the DAG in the above location
> >and
> >> display the specified file instead.
> >>
> >> I don't have any experience contributing to open source projects of
> >this
> >> size, would this be a welcome pull request? I'd rather not waste
> >anyone's
> >> time if it isn't.
> >>
> >> Thanks,
> >> Damian
> >>
> >>
> >===============================================================================
> >>
> >> Please access the attached hyperlink for an important electronic
> >> communications disclaimer:
> >> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
> >>
> >===============================================================================
> >>
> >>
>
>
> ===============================================================================
> Please access the attached hyperlink for an important electronic communications disclaimer:
> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
> ===============================================================================

Mime
View raw message