airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pawel Graczyk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-999) Support for Redis database
Date Tue, 23 Oct 2018 15:45:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660852#comment-16660852
] 

Pawel Graczyk commented on AIRFLOW-999:
---------------------------------------

Not sure if the way to go is to reopen this issue or to create new one, so I just put comment
for now...

Current implementation of AIRFLOW-999 needs corrections.

1. Password have to stay None in case there is no password set, otherwise AUTH call will
be send and that produces
error on redis without authorization set.
'''
self.password = None if (conn.password is None or str(conn.password) == 'None') else conn.password
'''
2. Calling conn = self.get_connection(self.redis_conn_id) (in constructor) leads to attaching
resource inside a hook
object as a reference. Please check how other hooks constructors look like. There is no references
to connections on
object initialisation.
2. Method key_exists is trivial and should be removed as it brings nothing.
3. This is less important but tests should mock redis lib instead of using it. We want to
test "our" code, so hook and
sensor, and not redis lib. Redis lib is part of airflow dependencies, and we e assume it works
well.

> Support for Redis database
> --------------------------
>
>                 Key: AIRFLOW-999
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-999
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: db
>            Reporter: msempere
>            Assignee: msempere
>            Priority: Minor
>              Labels: features
>             Fix For: 1.9.0
>
>
> Currently Airflow doesn't offer support for Redis DB.
> The idea is to create a Hook to connect to it and offer a minimal functionality.
> So the proposal is to create a sensor that monitor for a Redis key existence. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message