Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Mon, 6 Feb 2017 12:36:41 +0000 (UTC)
From: "Erik Cederstrand (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13040623.1486384503000.5083.1486384601721@Atlassian.JIRA>
In-Reply-To: <JIRA.13040623.1486384503000@Atlassian.JIRA>
References: <JIRA.13040623.1486384503000@Atlassian.JIRA> <JIRA.13040623.1486384503737@jira-lw-us.apache.org>
Subject: [jira] [Updated] (AIRFLOW-840) Python3 encoding issue in Kerberos
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 06 Feb 2017 12:36:46 -0000


     [ https://issues.apache.org/jira/browse/AIRFLOW-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erik Cederstrand updated AIRFLOW-840:
-------------------------------------
    Description: 
While attempting to configure Kerberos ticket renewal in a Python3 environment, I encountered this encoding issue trying to run {{airflow kerberos}}:

{quote}
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 15, in <module>
    args.func(args)
  File "/usr/local/lib/python3.4/dist-packages/airflow/bin/cli.py", line 600, in kerberos
    airflow.security.kerberos.run()
  File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py", line 110, in run
    renew_from_kt()
  File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py", line 55, in renew_from_kt
    "\n".join(subp.stderr.readlines())))
TypeError: sequence item 0: expected str instance, bytes found
{quote}

The issue here (ignoring for a moment why {{kinit}} is failing on my machine) is that Popen in Python3 returns {{bytes}} for stdin/stdout, but both are handled as if they are {{str}}.

I'm unsure what the Py2/3 compat policy is at Airflow, but a simple {{from six import PY2}} and an if/else seems like the least intrusive fix. The non-PY2 path would then add something like {{subp.stdin.readlines().decode(errors='ignore')}}

  was:
While attempting to configure Kerberos ticket renewal in a Python3 environment, I encountered this encoding issue trying to run {{airflow kerberos}}:

{{
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 15, in <module>
    args.func(args)
  File "/usr/local/lib/python3.4/dist-packages/airflow/bin/cli.py", line 600, in kerberos
    airflow.security.kerberos.run()
  File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py", line 110, in run
    renew_from_kt()
  File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py", line 55, in renew_from_kt
    "\n".join(subp.stderr.readlines())))
TypeError: sequence item 0: expected str instance, bytes found
}}

The issue here (ignoring for a moment why {{kinit}} is failing on my machine) is that Popen in Python3 returns {{bytes}} for stdin/stdout, but both are handled as if they are {{str}}.

I'm unsure what the Py2/3 compat policy is at Airflow, but a simple {{from six import PY2}} and an if/else seems like the least intrusive fix. The non-PY2 path would then add something like {{subp.stdin.readlines().decode(errors='ignore')}}


> Python3 encoding issue in Kerberos
> ----------------------------------
>
>                 Key: AIRFLOW-840
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-840
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: security
>    Affects Versions: Airflow 1.8
>         Environment: $ python --version
> Python 3.4.3
>            Reporter: Erik Cederstrand
>              Labels: security
>
> While attempting to configure Kerberos ticket renewal in a Python3 environment, I encountered this encoding issue trying to run {{airflow kerberos}}:
> {quote}
> Traceback (most recent call last):
>   File "/usr/local/bin/airflow", line 15, in <module>
>     args.func(args)
>   File "/usr/local/lib/python3.4/dist-packages/airflow/bin/cli.py", line 600, in kerberos
>     airflow.security.kerberos.run()
>   File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py", line 110, in run
>     renew_from_kt()
>   File "/usr/local/lib/python3.4/dist-packages/airflow/security/kerberos.py", line 55, in renew_from_kt
>     "\n".join(subp.stderr.readlines())))
> TypeError: sequence item 0: expected str instance, bytes found
> {quote}
> The issue here (ignoring for a moment why {{kinit}} is failing on my machine) is that Popen in Python3 returns {{bytes}} for stdin/stdout, but both are handled as if they are {{str}}.
> I'm unsure what the Py2/3 compat policy is at Airflow, but a simple {{from six import PY2}} and an if/else seems like the least intrusive fix. The non-PY2 path would then add something like {{subp.stdin.readlines().decode(errors='ignore')}}


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)