airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin McHale (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-2016) Add support for Dataproc Workflow Templates
Date Thu, 10 May 2018 17:03:00 GMT


Kevin McHale commented on AIRFLOW-2016:

hi [~DanSedov] I recently tried using these operators (or, at least, the instantiate-inline
variant) and I received errors due to the {{template_id}} field being missing.  I think that
the issue comes from the airflow {{template_fields}} settings that are set on [this line|] in
the base class.  Airflow's jinja templater tries to populate these fields on the derived
classes, but {{template_id}} does not apply for the instantiate-inline operator, and likewise
{{template}} should not apply for the instantiate operator, so an error is thrown.

I have put together a quick fix that looks like [this|] and
I am running that successfully on our Airflow deployments at Etsy, but I just wanted to check
with you before submitting it to make sure I'm not missing something?  In particular we are
using Airflow 1.9.0 and I suppose it's possible that the jinja templater behaves differently
in the latest airflow versions?

> Add support for Dataproc Workflow Templates
> -------------------------------------------
>                 Key: AIRFLOW-2016
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib
>            Reporter: Dan Sedov
>            Assignee: Dan Sedov
>            Priority: Minor
>             Fix For: 1.10.0
> Add new operators to support instantiation of Google Cloud Dataproc Workflow Templates.
> See:

This message was sent by Atlassian JIRA

View raw message