Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Wed, 11 Oct 2017 00:18:01 +0000 (UTC)
From: "Joy Gao (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13102370.1505420120000.23844.1507681081007@Atlassian.JIRA>
In-Reply-To: <JIRA.13102370.1505420120000@Atlassian.JIRA>
References: <JIRA.13102370.1505420120000@Atlassian.JIRA> <JIRA.13102370.1505420120483@jira-lw-us.apache.org>
Subject: [jira] [Updated] (AIRFLOW-1613) Make
 MySqlToGoogleCloudStorageOperator compaitible with python3
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 11 Oct 2017 00:18:11 -0000


     [ https://issues.apache.org/jira/browse/AIRFLOW-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joy Gao updated AIRFLOW-1613:
-----------------------------
    Description: 
1. 
In Python 3, map(...) returns an iterator, which can only be iterated over once. 
Therefore the current implementation will return an empty list after the first iteration of schema:

{code}
        schema = map(lambda schema_tuple: schema_tuple[0], cursor.description)
        file_no = 0
        tmp_file_handle = NamedTemporaryFile(delete=True)
        tmp_file_handles = {self.filename.format(file_no): tmp_file_handle}

        for row in cursor:
            # Convert datetime objects to utc seconds, and decimals to floats
            row = map(self.convert_types, row)
            row_dict = dict(zip(schema, row))
{code}

2.
File opened as binary, but string are written to it. Get error `a bytes-like object is required, not 'str'`. Use mode='w' instead.

3. Update:
Currently All 

  was:
1. 
In Python 3, map(...) returns an iterator, which can only be iterated over once. 
Therefore the current implementation will return an empty list after the first iteration of schema:

{code}
        schema = map(lambda schema_tuple: schema_tuple[0], cursor.description)
        file_no = 0
        tmp_file_handle = NamedTemporaryFile(delete=True)
        tmp_file_handles = {self.filename.format(file_no): tmp_file_handle}

        for row in cursor:
            # Convert datetime objects to utc seconds, and decimals to floats
            row = map(self.convert_types, row)
            row_dict = dict(zip(schema, row))
{code}

2.
File opened as binary, but string are written to it. Get error `a bytes-like object is required, not 'str'`. Use mode='w' instead.


> Make MySqlToGoogleCloudStorageOperator compaitible with python3
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-1613
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1613
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib
>            Reporter: Joy Gao
>            Assignee: Joy Gao
>             Fix For: 1.9.0
>
>
> 1. 
> In Python 3, map(...) returns an iterator, which can only be iterated over once. 
> Therefore the current implementation will return an empty list after the first iteration of schema:
> {code}
>         schema = map(lambda schema_tuple: schema_tuple[0], cursor.description)
>         file_no = 0
>         tmp_file_handle = NamedTemporaryFile(delete=True)
>         tmp_file_handles = {self.filename.format(file_no): tmp_file_handle}
>         for row in cursor:
>             # Convert datetime objects to utc seconds, and decimals to floats
>             row = map(self.convert_types, row)
>             row_dict = dict(zip(schema, row))
> {code}
> 2.
> File opened as binary, but string are written to it. Get error `a bytes-like object is required, not 'str'`. Use mode='w' instead.
> 3. Update:
> Currently All 


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)