airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bolke de Bruin (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-1780) Long Unicode Characters In BashOperator Output Cause Task Hang
Date Tue, 07 Nov 2017 19:06:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bolke de Bruin resolved AIRFLOW-1780.
-------------------------------------
    Resolution: Fixed

> Long Unicode Characters In BashOperator Output Cause Task Hang
> --------------------------------------------------------------
>
>                 Key: AIRFLOW-1780
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1780
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Dan Davydov
>            Assignee: Dan Davydov
>             Fix For: 1.9.1
>
>
> Conditions to replicate:
> Create a DAG with a single BashOperator that logs ~10 long lines of text with at least
one unicode character (e.g. cat a file with these contents)
> Use BashTaskRunner
> Log to a file
> Run a worker that picks up the task or an airflow run --local command
> Behavior:
> The BashOperator/cat command hangs
> Most likely this is due to a pipe issue, where the unicode characters are filling up
the pipe and .format() is not able to process partial unicode characters. Interestingly removing
one of the conditions above (e.g. logging to stdout instead of a file or having short lines
doesn't cause the issue).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message