airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Kotecha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2759) Simplify proxy server based access to external platforms like Google cloud
Date Mon, 23 Jul 2018 10:04:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552581#comment-16552581
] 

Ravi Kotecha commented on AIRFLOW-2759:
---------------------------------------

@[~aimohan] it looks like httplib2 can use HTTP_PROXY and HTTPS_PROXY environment variables
to set a proxy so it's probably best to just do that.

https://github.com/httplib2/httplib2/blob/10909e980477578f909b1e164476d9149ff3581e/tests/test_proxy.py#L34

> Simplify proxy server based access to external platforms like Google cloud 
> ---------------------------------------------------------------------------
>
>                 Key: AIRFLOW-2759
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2759
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: hooks
>            Reporter: Aishwarya Mohan
>            Assignee: Aishwarya Mohan
>            Priority: Major
>              Labels: hooks, proxy
>
> Several companies adopt a Proxy Server based approach in order to provide an additional
layer of security while communicating with external platforms to establish legitimacy of caller
and calle. A potential use case would be writing logs from Airflow to a cloud storage platform
like google cloud via an intermediary proxy server.
> In the current scenario the proxy details need to be hardcoded and passed to the HTTP
client library(httplib2) in the GoogleCloudBaseHook class (gcp_api_base_hook.py). It would
be convenient if the proxy details (for example, host and port) can be extracted from the
airflow configuration file as opposed to hardcoding the details at hook level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message