From commits-return-23675-archive-asf-public=cust-asf.ponee.io@airflow.incubator.apache.org Tue Sep 25 11:36:06 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 68D7418061A for ; Tue, 25 Sep 2018 11:36:05 +0200 (CEST) Received: (qmail 89269 invoked by uid 500); 25 Sep 2018 09:36:04 -0000 Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list commits@airflow.incubator.apache.org Received: (qmail 89260 invoked by uid 99); 25 Sep 2018 09:36:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Sep 2018 09:36:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 153BB18560E for ; Tue, 25 Sep 2018 09:36:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id i8Mk8BgFl_4U for ; Tue, 25 Sep 2018 09:36:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 4E2B85F124 for ; Tue, 25 Sep 2018 09:36:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id AB22AE0F4C for ; Tue, 25 Sep 2018 09:36:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 5A2FB23FA1 for ; Tue, 25 Sep 2018 09:36:00 +0000 (UTC) Date: Tue, 25 Sep 2018 09:36:00 +0000 (UTC) From: "Minh Quan TRAN (JIRA)" To: commits@airflow.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (AIRFLOW-3093) Airflow UnicodeEncodeError with logging (file_task_handler.py) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/AIRFLOW-3093?page=3Dcom.atlass= ian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Quan TRAN updated AIRFLOW-3093: ------------------------------------ Description:=20 I ran into this exception stack when using BashOperator along with docker &= composer install inside the container. I ran some more tests with some oth= er set of dependencies, this error is not always present, just for some cas= es the exception is thrown. =C2=A0 The exception is thrown at=C2=A0*file_task_handler.py* =C2=A0 {code:java} Python 3.5{code} {code:java} [2018-09-19 20:42:18,708] {bash_operator.py:111} INFO - Package operations:= 134 installs, 0 updates, 0 removals [2018-09-19 20:42:18,790] {bash_operator.py:111} INFO - - Installing ocra= mius/package-versions (1.3.0): Downloading (100%) [2018-09-19 20:42:18,850] {bash_operator.py:111} INFO - - Installing symf= ony/flex (v1.1.1): Downloading (100%) [2018-09-19 20:42:18,897] {bash_operator.py:111} INFO -=20 [2018-09-19 16:12:51,554] {logging_mixin.py:95} WARNING - --- Logging error= --- [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Traceback (most r= ecent call last): [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 983, in emit stream.write(msg) [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - UnicodeEncodeErro= r: 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordina= l not in range(128) [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Call stack: [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/loca= l/bin/airflow", line 32, in args.func(args) [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/utils/cli.py", line 74, in wrapper return f(*args, **kwargs) [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/bin/cli.py", line 498, in run _run(args, dag, ti) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/bin/cli.py", line 402, in _run pool=3Dargs.pool, [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/utils/db.py", line 74, in wrapper return func(*args, **kwargs) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/models.py", line 1633, in _run_raw_ta= sk result =3D task_copy.execute(context=3Dcontext) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/operators/bash_operator.py", line 110= , in execute self.log.info(line) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1280, in info self._log(INFO, msg, args, **kwargs) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1416, in _log self.handle(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1426, in handle self.callHandlers(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1488, in callHandlers hdlr.handle(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 856, in handle self.emit(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/utils/log/file_task_handler.py", line= 61, in emit self.handler.emit(record){code} {code:java} Python 3.6{code} {code:java} [2018-09-25 09:33:04,622] {logging_mixin.py:95} WARNING - --- Logging error= --- [2018-09-25 09:33:04,627] {logging_mixin.py:95} WARNING - Traceback (most r= ecent call last): [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.6/logging/__init__.py", line 995, in emit stream.write(msg) [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - UnicodeEncodeErro= r: 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordina= l not in range(128) [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - Call stack: [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/bin/airflow", line 32, in args.func(args) [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/utils/cli.py", line 74, in wrapper return f(*args, **kwargs) [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/bin/cli.py", line 498, in run _run(args, dag, ti) [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/bin/cli.py", line 402, in _run pool=3Dargs.pool, [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/utils/db.py", line 74, in wrapper return func(*args, **kwargs) [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/models.py", line 1633, in _run_raw_ta= sk result =3D task_copy.execute(context=3Dcontext) [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/operators/bash_operator.py", line 110= , in execute self.log.info(line) [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.6/logging/__init__.py", line 1307, in info self._log(INFO, msg, args, **kwargs) [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.6/logging/__init__.py", line 1443, in _log self.handle(record) [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.6/logging/__init__.py", line 1453, in handle self.callHandlers(record) [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.6/logging/__init__.py", line 1515, in callHandlers hdlr.handle(record) [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.6/logging/__init__.py", line 864, in handle self.emit(record) [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.6/dist-packages/airflow/utils/log/file_task_handler.py", line= 61, in emit self.handler.emit(record){code} was: I ran into this exception stack when using BashOperator along with docker &= composer install inside the container. I ran some more tests with some oth= er set of dependencies, this error is not always present, just for some cas= es the exception is thrown. =C2=A0 The exception is thrown at=C2=A0*file_task_handler.py* =C2=A0 {code:java} [2018-09-19 20:42:18,708] {bash_operator.py:111} INFO - Package operations:= 134 installs, 0 updates, 0 removals [2018-09-19 20:42:18,790] {bash_operator.py:111} INFO - - Installing ocra= mius/package-versions (1.3.0): Downloading (100%) [2018-09-19 20:42:18,850] {bash_operator.py:111} INFO - - Installing symf= ony/flex (v1.1.1): Downloading (100%) [2018-09-19 20:42:18,897] {bash_operator.py:111} INFO -=20 [2018-09-19 16:12:51,554] {logging_mixin.py:95} WARNING - --- Logging error= --- [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Traceback (most r= ecent call last): [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 983, in emit stream.write(msg) [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - UnicodeEncodeErro= r: 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordina= l not in range(128) [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Call stack: [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/loca= l/bin/airflow", line 32, in args.func(args) [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/utils/cli.py", line 74, in wrapper return f(*args, **kwargs) [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/bin/cli.py", line 498, in run _run(args, dag, ti) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/bin/cli.py", line 402, in _run pool=3Dargs.pool, [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/utils/db.py", line 74, in wrapper return func(*args, **kwargs) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/models.py", line 1633, in _run_raw_ta= sk result =3D task_copy.execute(context=3Dcontext) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/operators/bash_operator.py", line 110= , in execute self.log.info(line) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1280, in info self._log(INFO, msg, args, **kwargs) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1416, in _log self.handle(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1426, in handle self.callHandlers(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 1488, in callHandlers hdlr.handle(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lib/= python3.5/logging/__init__.py", line 856, in handle self.emit(record) [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/loca= l/lib/python3.5/dist-packages/airflow/utils/log/file_task_handler.py", line= 61, in emit self.handler.emit(record){code} > Airflow UnicodeEncodeError with logging (file_task_handler.py) > -------------------------------------------------------------- > > Key: AIRFLOW-3093 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3093 > Project: Apache Airflow > Issue Type: Bug > Components: logging > Affects Versions: 1.10.0 > Reporter: Minh Quan TRAN > Priority: Major > Labels: encoding > > I ran into this exception stack when using BashOperator along with docker= & composer install inside the container. I ran some more tests with some o= ther set of dependencies, this error is not always present, just for some c= ases the exception is thrown. > =C2=A0 > The exception is thrown at=C2=A0*file_task_handler.py* > =C2=A0 > {code:java} > Python 3.5{code} > {code:java} > [2018-09-19 20:42:18,708] {bash_operator.py:111} INFO - Package operation= s: 134 installs, 0 updates, 0 removals > [2018-09-19 20:42:18,790] {bash_operator.py:111} INFO - - Installing oc= ramius/package-versions (1.3.0): Downloading (100%) > [2018-09-19 20:42:18,850] {bash_operator.py:111} INFO - - Installing sy= mfony/flex (v1.1.1): Downloading (100%) > [2018-09-19 20:42:18,897] {bash_operator.py:111} INFO -=20 > [2018-09-19 16:12:51,554] {logging_mixin.py:95} WARNING - --- Logging err= or --- > [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Traceback (most= recent call last): > [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.5/logging/__init__.py", line 983, in emit > stream.write(msg) > [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - UnicodeEncodeEr= ror: 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordi= nal not in range(128) > [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Call stack: > [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/bin/airflow", line 32, in > args.func(args) > [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/utils/cli.py", line 74, in wrapper > return f(*args, **kwargs) > [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/bin/cli.py", line 498, in run > _run(args, dag, ti) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/bin/cli.py", line 402, in _run > pool=3Dargs.pool, > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/utils/db.py", line 74, in wrapper > return func(*args, **kwargs) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/models.py", line 1633, in _run_raw_= task > result =3D task_copy.execute(context=3Dcontext) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/operators/bash_operator.py", line 1= 10, in execute > self.log.info(line) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.5/logging/__init__.py", line 1280, in info > self._log(INFO, msg, args, **kwargs) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.5/logging/__init__.py", line 1416, in _log > self.handle(record) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.5/logging/__init__.py", line 1426, in handle > self.callHandlers(record) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.5/logging/__init__.py", line 1488, in callHandlers > hdlr.handle(record) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.5/logging/__init__.py", line 856, in handle > self.emit(record) > [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.5/dist-packages/airflow/utils/log/file_task_handler.py", li= ne 61, in emit > self.handler.emit(record){code} > {code:java} > Python 3.6{code} > {code:java} > [2018-09-25 09:33:04,622] {logging_mixin.py:95} WARNING - --- Logging err= or --- > [2018-09-25 09:33:04,627] {logging_mixin.py:95} WARNING - Traceback (most= recent call last): > [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.6/logging/__init__.py", line 995, in emit > stream.write(msg) > [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - UnicodeEncodeEr= ror: 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordi= nal not in range(128) > [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - Call stack: > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/bin/airflow", line 32, in > args.func(args) > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/utils/cli.py", line 74, in wrapper > return f(*args, **kwargs) > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/bin/cli.py", line 498, in run > _run(args, dag, ti) > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/bin/cli.py", line 402, in _run > pool=3Dargs.pool, > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/utils/db.py", line 74, in wrapper > return func(*args, **kwargs) > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/models.py", line 1633, in _run_raw_= task > result =3D task_copy.execute(context=3Dcontext) > [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/operators/bash_operator.py", line 1= 10, in execute > self.log.info(line) > [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.6/logging/__init__.py", line 1307, in info > self._log(INFO, msg, args, **kwargs) > [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.6/logging/__init__.py", line 1443, in _log > self.handle(record) > [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.6/logging/__init__.py", line 1453, in handle > self.callHandlers(record) > [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.6/logging/__init__.py", line 1515, in callHandlers > hdlr.handle(record) > [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/li= b/python3.6/logging/__init__.py", line 864, in handle > self.emit(record) > [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File "/usr/lo= cal/lib/python3.6/dist-packages/airflow/utils/log/file_task_handler.py", li= ne 61, in emit > self.handler.emit(record){code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)