airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Breame (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-103) Allow jinja templates to be used in task params
Date Wed, 14 Jun 2017 12:19:00 GMT


Colin Breame commented on AIRFLOW-103:

Totally agree that this is a common use case.

For example, you might have something like this:

{code}op = BashOperator(
	task_id = "task",
	bash_command = " {{params.path}}",
        params = {"path": "/path/to/thing/{{ds}}/thing.csv"},
        dag = dag

In this case, the bash_command will be expanded to:

{code} /path/to/thing/{{ds}}/things.csv

But should be expanded to:

{code} /path/to/thing/2017-06-14/things.csv

> Allow jinja templates to be used in task params
> -----------------------------------------------
>                 Key: AIRFLOW-103
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: operators
>            Reporter: Paul Rhodes
>            Priority: Minor
> At present when defining an Operator it isn't possible to reference template variables
such as {{ds}} in the params block.
> Enabling this makes it easier to reuse Operators with very similar configurations. An
example of this might be a BashOperator which runs a large (possibly external) script that
only has a couple of configuration changes used between different runs.

This message was sent by Atlassian JIRA

View raw message