airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Reda (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-240) TemplateNotFound Error when using PostgresOperator and SQL script
Date Tue, 14 Jun 2016 21:33:30 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330664#comment-15330664
] 

Greg Reda commented on AIRFLOW-240:
-----------------------------------

Threw this into a Stack Overflow question in hopes of making it more easily Google-able: http://stackoverflow.com/questions/37822380/templatenotfound-when-using-airflows-postgresoperator-with-jinja-templating-and

> TemplateNotFound Error when using PostgresOperator and SQL script
> -----------------------------------------------------------------
>
>                 Key: AIRFLOW-240
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-240
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>         Environment: Ubuntu 14.04
>            Reporter: Greg Reda
>
> When using the PostgresOperator, I've been unable to get templating to work. It's quite
possible I'm doing something wrong, but I'm pretty lost as to what the issue might be. Here's
an example to reproduce the TemplateNotFound error I've been getting:
> {code:title=/my_dag/csv_to_redshift.py}
> copy_s3_to_redshift = PostgresOperator(
>     task_id='load_table',
>     sql=this_dag_path + '/copy_to_redshift.sql',
>     params=dict(
>         AWS_ACCESS_KEY_ID=Variable.get('AWS_ACCESS_KEY_ID'),
>         AWS_SECRET_ACCESS_KEY=Variable.get('AWS_SECRET_ACCESS_KEY')
>     ),
>     postgres_conn_id='postgres_redshift',
>     autocommit=False,
>     dag=dag
> )
> {code}
> {code:title=/my_dag/copy_to_redshift.sql}
> COPY public.table_foobar FROM 's3://mybucket/test-data/import/foobar.csv'
> CREDENTIALS 'aws_access_key_id={{ AWS_ACCESS_KEY_ID }};aws_secret_access_key={{ AWS_SECRET_ACCESS_KEY
}}'
> CSV
> NULL as 'null'
> IGNOREHEADER as 1;
> {code}
> {code}
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 2038, in resolve_template_files
>     setattr(self, attr, env.loader.get_source(env, content)[0])
>   File "/usr/local/lib/python2.7/dist-packages/jinja2/loaders.py", line 187, in get_source
>     raise TemplateNotFound(template)
> TemplateNotFound: /home/gregreda/airflow/dags/my_dag/copy_to_redshift.sql
> {code}
> Any ideas towards a fix are much appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message