airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Custom DAG name?
Date Wed, 03 May 2017 15:21:08 GMT
You could add the run_id to the DagRun list view, would that help?

Max

On Wed, May 3, 2017 at 8:02 AM, Paul Zaczkiewicz <paulzacz@gmail.com> wrote:

> Sounds like an interesting application of airflow which may not be the best
> use-case for it.
>
> How are you triggering these dag runs? airflow trigger_dag? If so, you can
> find the logs for your specific RUN_ID in the ui without grepping through
> all logs first, but with hundreds of dag runs, it will still take a bit of
> searching. If the unique ID is something known internally to your
> application, then it's not something that Airflow knows about, so grep is
> your only recourse.
>
> On Wed, May 3, 2017 at 9:05 AM, EKC (Erik Cederstrand) <EKC@novozymes.com>
> wrote:
>
> > Hi all,
> >
> >
> > I have created a DAG, 'my_dag', that will always be triggered manually,
> > and will always receive a unique ID from the command-line, specifying the
> > entity the DAG must process. The DAG may be triggered hundreds of times a
> > day.
> >
> >
> > In the Airflow UI, the DAG will show up everywhere as 'my_dag', which is
> > not very relevant to me. A single DAG run may run for several days, so I
> > want to easily look up specific DAG runs using the ID, to check the
> status
> > of that DAG run. But the ID only shows up in the log file, so to find the
> > status of a specific DAG run, I need to open every DAG run and check the
> > log file. That is very cumbersome when there are hundreds of simultaneous
> > DAG runs.
> >
> >
> > What are my options if I want to improve this situation?
> >
> >
> > Kind regards,
> >
> > Erik
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message