airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yohei Onishi <yohei.onish...@gmail.com>
Subject Connect to TLS-enabled Elasticache Redis from Airflow Celery Executor
Date Mon, 21 Sep 2020 15:19:06 GMT
Hi,

I am currently using Celery Executor on Airflow 1.10. My broker is AWS
Elasticache Redis (v. 5.0.6). How can I enable Encryption in-transit?

According to Airflow source code, ssl_keyfile, ssl_certfile and
ssl_ca_certs are required.
https://github.com/apache/airflow/blob/1.10.10/airflow/config_templates/default_celery.py#L68-L72

elif 'redis://' in broker_url:
>     broker_use_ssl = {'ssl_keyfile': conf.get('celery', 'SSL_KEY'),
>                       'ssl_certfile': conf.get('celery', 'SSL_CERT'),
>                       'ssl_ca_certs': conf.get('celery', 'SSL_CACERT'),
>                       'ssl_cert_reqs': ssl.CERT_REQUIRED}


But Elasticache Redis does not provide those TLS certificate. The official
document only explains a solution for redis-cli, which uses TLS tunnelling
by stunnel.

https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/in-transit-encryption.html#connect-tls

Does Airflow Celery Executor support Encryption in-transit on Elasticache
Redis? If so, how can we implement it?

I have also posted the same questions.
https://stackoverflow.com/questions/63995039/connect-to-tls-enabled-elasticache-redis-from-airflow-celery-executor

Mime
View raw message