airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Good jinja templating in airflow examples
Date Thu, 19 Jan 2017 03:09:54 GMT
Here's the list of variables and macros exposed in the jinja context:
https://airflow.apache.org/code.html#macros

Those are all exposed by the framework, meaning they are usable for any
templated field without doing anything special.

There are ways to pass your own variables and methods using the `params`
attribute of any operator as shown in the tutorial here
<https://airflow.apache.org/tutorial.html#example-pipeline-definition>, and
there's also a `user_defined_macros` parameter as a dict you can pass while
creating the DAG object. Every key in that dictionary is made available in
the global jinja namespace.

https://airflow.apache.org/code.html#airflow.models.DAG

Max

On Wed, Jan 18, 2017 at 11:31 AM, Boris Tyukin <boris@boristyukin.com>
wrote:

> Hi Guilherme,
>
> I guess it depends what exactly you want to do as not everything works with
> jinja.
>
> From documentation:
>
> https://pythonhosted.org/airflow/concepts.html#jinja-templating
>
> You can use Jinja templating with every parameter that is marked as
> “templated” in the documentation.
>
> You can open source code for operators and see what parameters are actually
> templated.
>
> For example, if you open source code for BashOperator
> https://pythonhosted.org/airflow/_modules/bash_operator.html#BashOperator
> you will see this line of code:
> template_fields = ('bash_command', 'env')
>
> It means only bash_command and env will work with jinja templates.
>
> On Wed, Jan 18, 2017 at 12:45 PM, Guilherme Marthe <
> guilherme.marthe@enjoei.com.br> wrote:
>
> > Hey folks!
> >
> > I am trying to write a dag that works well with jinja templating, since
> > through my study of the documentation, is the best way to ensure
> > compatibility with the back-fill function.
> >
> > Are you guys aware of any examples online with this functionality
> working?
> > The, documentation is still a bit dry on insightful examples, and I am a
> > noobie developer, so I am trying to make the best use of the tool in the
> > "correct way".
> >
> > Thank you in advance for any resources you guys can share!
> >
> > Gui
> >
> > Ps: I ve been told I can email you guys with questions like these :) Hope
> > this is not a hassle ^_^
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message