airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Chamarthi <rajesh.chamar...@gmail.com>
Subject Environment variables - Oracle LD_LIBRARY_PATH
Date Fri, 08 Dec 2017 15:55:04 GMT
Hi all,

What is the right way to set environment variables to be used in DAGS. 
Also, is it possible to export them such that currently running scheduler and worker processes
can use it?

Context :

I am working on a DAG that involves making an Oracle connection, which requires the LD_LIBRARY_PATH
variable to be set. I added it to /etc/profile.d of the airflow user. When I start a new shell
+ python virtualenv and run “airflow test dag_name task_name”, the task is able to use
the path variable above and make the necessary connection. 

However, when it runs through the scheduler, the task fails with the following error : “Oracle
client library cannot be loaded”, presumably because it cannot read the LD_LIBRARY_PATH.

2) I have also tried the following code in my custom hook, to explicitly set the path. I see
the same error.

if 'LD_LIBRARY_PATH' not in os.environ.keys():
    os.environ['LD_LIBRARY_PATH']="/opt/oracle/instantclient_11_2"
    logging.info("added ld library path", os.environ['LD_LIBRARY_PATH'])

Any pointers on how I can resolve this? 

Thanks,
Rajesh


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message