airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon McKenzie <jcmc...@gmail.com>
Subject Dynamic DAG Input/Seed Data?
Date Mon, 11 Jul 2016 21:46:46 GMT
Unless I'm missing it, it appears like it isn't possible to launch a DAG
job with initial inputs to the first task instance in the workflow (without
specifying those inputs in the DAG definition)

Am I missing something?

So for instance, I want to have user A be able to launch the DAG with
parameter foo = bar, and user B to be able to launch the same DAG with foo
= baz. In my use case, this would be hooked up to a RESTful API, and the
users wouldn't necessarily know anything about DAGs or what's happening
behind the scenes

The closest I can think to accomplishing this is to generate run IDs in my
REST API, store the (run ID, input) pair in a database, and retrieve the
inputs in my first task in my DAG. But this seems like a very hamhanded,
roundabout way of doing it. I'd much rather just create a DagRun with
task_params that the scheduler automatically associates to the first task
instance.

Any thoughts?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message