Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Mon, 17 Oct 2016 04:22:58 +0000 (UTC)
From: "Michael Shire (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13012668.1476604307000.938.1476678178338@Atlassian.JIRA>
In-Reply-To: <JIRA.13012668.1476604307000@Atlassian.JIRA>
References: <JIRA.13012668.1476604307000@Atlassian.JIRA> <JIRA.13012668.1476604307054@arcas>
Subject: [jira] [Updated] (AIRFLOW-573) Jinja2 template failing env argument
 in BashOperater if string ends in .sh or .bash
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 17 Oct 2016 04:23:04 -0000


     [ https://issues.apache.org/jira/browse/AIRFLOW-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Shire updated AIRFLOW-573:
----------------------------------
    Summary: Jinja2 template failing env argument in BashOperater if string ends in .sh or .bash  (was: Jinja2 template not rendering if the string has a period/dot in it)

> Jinja2 template failing env argument in BashOperater if string ends in .sh or .bash
> -----------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-573
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-573
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>    Affects Versions: Airflow 2.0
>            Reporter: Michael Shire
>
> A dot/period in a templated string appears to cause jinja template errors in Airflow.
> example:
> I constructed a dictionary to pass as an environment into BashOperator
> i.e.
> envvars = {'use_this_script':'/Users/mshire/x.sh'}
> task4 = BashOperator(
>     bash_command="env",
>     task_id="test_env_variables",
>     env=envvars,
>     dag=dag
>     )
> If I remove the dot/period (i.e. ".") in "x.sh" then it works.  Otherwise I get an error below.  I have tried it directly in python with jinja2 and it works, but not in Airflow.
> > airflow test scratch test_env_variables 2016-10-01
> [2016-10-16 00:39:32,981] {__init__.py:36} INFO - Using executor SequentialExecutor
> [2016-10-16 00:39:33,374] {models.py:154} INFO - Filling up the DagBag from /Users/mshire/airflow/dags
> envvars= {'use_this_script': '/Users/mshire/x.sh'}
> [2016-10-16 00:39:33,479] {models.py:1196} INFO -
> --------------------------------------------------------------------------------
> Starting attempt 1 of 1
> --------------------------------------------------------------------------------
> [2016-10-16 00:39:33,480] {models.py:1219} INFO - Executing <Task(BashOperator): test_env_variables> on 2016-10-01 00:00:00
> [2016-10-16 00:39:33,489] {models.py:1286} ERROR - /Users/mshire/x.sh
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1233, in run
>     self.render_templates()
>   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1409, in render_templates
>     rendered_content = rt(attr, content, jinja_context)
>   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 2019, in render_template
>     return self.render_template_from_field(attr, content, context, jinja_env)
>   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1995, in render_template_from_field
>     for k, v in list(content.items())}
>   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1995, in <dictcomp>
>     for k, v in list(content.items())}
>   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 2017, in render_template
>     return jinja_env.get_template(content).render(**context)
>   File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 812, in get_template
>     return self._load_template(name, self.make_globals(globals))
>   File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 774, in _load_template
>     cache_key = self.loader.get_source(self, name)[1]
>   File "/usr/local/lib/python2.7/site-packages/jinja2/loaders.py", line 187, in get_source
>     raise TemplateNotFound(template)
> TemplateNotFound: /Users/mshire/x.sh
> [2016-10-16 00:39:33,490] {models.py:1306} INFO - Marking task as FAILED.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)