airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahas L (Jira)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-6208) StreamLogWriter class does not implement file fileno() function, preventing output redirection to stdout with the python subprocess module
Date Tue, 10 Dec 2019 00:55:00 GMT
Sahas L created AIRFLOW-6208:
--------------------------------

             Summary: StreamLogWriter class does not implement file fileno() function, preventing
output redirection to stdout with the python subprocess module
                 Key: AIRFLOW-6208
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6208
             Project: Apache Airflow
          Issue Type: Bug
          Components: logging
    Affects Versions: 1.10.6
         Environment: Archlinux python 3.8 Linux 5.4
Also affects airflow breeze python 3.6 docker development environment
            Reporter: Sahas L
            Assignee: Sahas L


When using the python operator to execute a callbackfunction which makes use of the subprocess
module stdout redirection to system stdout, the task fails due to fileno() being unimplemented
in airflow/utils/log/logging_mixin.py class StreamLogWriter.
{code:java}
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 930, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/lib/python3.8/site-packages/airflow/operators/python_operator.py", line 113,
in execute
    return_value = self.execute_callable()
  File "/usr/lib/python3.8/site-packages/airflow/operators/python_operator.py", line 118,
in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/home/sahas/airflow/dags/subprocess.py", line 39, in sp
    subprocess.run("echo 1", shell=True, check=True, stdout=sys.stdout, stderr=sys.stderr)
  File "/usr/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.8/subprocess.py", line 804, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib/python3.8/subprocess.py", line 1487, in _get_handles
    c2pwrite = stdout.fileno()
AttributeError: 'StreamLogWriter' object has no attribute 'fileno'
{code}
The code in the callback function is:
{code:java}
import sys
import os

subprocess.run("echo 1", shell=True, check=True, stdout=sys.stdout, stderr=sys.stderr)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message