airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Georg Walther (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-447) Python 3 map object cannot be json-serialized - need lists instead
Date Thu, 01 Sep 2016 13:44:20 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Georg Walther resolved AIRFLOW-447.
-----------------------------------
    Resolution: Fixed

This issue has been resolved.

> Python 3 map object cannot be json-serialized - need lists instead
> ------------------------------------------------------------------
>
>                 Key: AIRFLOW-447
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-447
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib
>    Affects Versions: Airflow 1.7.1.3
>         Environment: Python 3
>            Reporter: Georg Walther
>            Priority: Blocker
>              Labels: easyfix
>
> On this line, `source_uris` are generated with `map`:
> https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/gcs_to_bq.py#L124
> In Python 2 `map` would return a list, however in Python 3 `map` returns a `map` object
/ generator.
> The Python 3 `map` object cannot be json-serialized downstream when storing corresponding
job objects in the database:
> https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L441
> Constructing `source_uris` with a list comprehension instead of `map` resolves the issue
for Python 3 and retains compatibility with Python 2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message