airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Zaczkiewicz <paulz...@gmail.com>
Subject Re: Custom DAG name?
Date Wed, 03 May 2017 15:02:08 GMT
Sounds like an interesting application of airflow which may not be the best
use-case for it.

How are you triggering these dag runs? airflow trigger_dag? If so, you can
find the logs for your specific RUN_ID in the ui without grepping through
all logs first, but with hundreds of dag runs, it will still take a bit of
searching. If the unique ID is something known internally to your
application, then it's not something that Airflow knows about, so grep is
your only recourse.

On Wed, May 3, 2017 at 9:05 AM, EKC (Erik Cederstrand) <EKC@novozymes.com>
wrote:

> Hi all,
>
>
> I have created a DAG, 'my_dag', that will always be triggered manually,
> and will always receive a unique ID from the command-line, specifying the
> entity the DAG must process. The DAG may be triggered hundreds of times a
> day.
>
>
> In the Airflow UI, the DAG will show up everywhere as 'my_dag', which is
> not very relevant to me. A single DAG run may run for several days, so I
> want to easily look up specific DAG runs using the ID, to check the status
> of that DAG run. But the ID only shows up in the log file, so to find the
> status of a specific DAG run, I need to open every DAG run and check the
> log file. That is very cumbersome when there are hundreds of simultaneous
> DAG runs.
>
>
> What are my options if I want to improve this situation?
>
>
> Kind regards,
>
> Erik
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message