airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Diane Ivy (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (AIRFLOW-2118) get_pandas_df does always pass a list of rows to be parsed
Date Tue, 06 Mar 2018 01:03:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Diane Ivy closed AIRFLOW-2118.
------------------------------
    Resolution: Fixed

Fixed with https://github.com/apache/incubator-airflow/pull/3066

> get_pandas_df does always pass a list of rows to be parsed
> ----------------------------------------------------------
>
>                 Key: AIRFLOW-2118
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2118
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib, hooks
>    Affects Versions: 1.9.0
>         Environment: pandas-gbp 0.3.1
>            Reporter: Diane Ivy
>            Assignee: Diane Ivy
>            Priority: Minor
>              Labels: easyfix
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> While trying to parse the pages in get_pandas_df if only one page is returned it starts
popping off each row and then the gbq_parse_data works incorrectly.
> {{while len(pages) > 0:}}
> {{    page = pages.pop()}}
> {{    dataframe_list.append(gbq_parse_data(schema, page))}}
> Possible solution:
> {{from google.cloud import bigquery}}
> {{if isinstance(pages[0], bigquery.table.Row):}}
> {{    pages = [pages]}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message