Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 72BF9200CD1 for ; Wed, 26 Jul 2017 13:33:21 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6FBD2168ADF; Wed, 26 Jul 2017 11:33:21 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 68A9B168ADE for ; Wed, 26 Jul 2017 13:33:20 +0200 (CEST) Received: (qmail 47913 invoked by uid 500); 26 Jul 2017 11:33:19 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 47903 invoked by uid 99); 26 Jul 2017 11:33:18 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Jul 2017 11:33:18 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 4EE96180812 for ; Wed, 26 Jul 2017 11:33:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.83 X-Spam-Level: *** X-Spam-Status: No, score=3.83 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_FONT_FACE_BAD=0.289, HTML_MESSAGE=2, HTML_OBFUSCATE_10_20=1.162, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id K0-kOT-SfVqB for ; Wed, 26 Jul 2017 11:33:16 +0000 (UTC) Received: from mail-vk0-f45.google.com (mail-vk0-f45.google.com [209.85.213.45]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id D6FEF5FC9D for ; Wed, 26 Jul 2017 11:33:15 +0000 (UTC) Received: by mail-vk0-f45.google.com with SMTP id r199so8475890vke.4 for ; Wed, 26 Jul 2017 04:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9nuQ1A5wW49KkD24auT2toreqMCyriAoldfaGpol7FQ=; b=DZBEh2w6kPivCfjJSory3KgJLjI98v8duso9PHM/i6OdurSB68ov9qpRjcCQcNcHah 51FxZqoyFonX33sZssJDE2g086BbZiXfjax+IDh2nrsy+UkXocW/osw0QxOq0C/3Rngx wVUtN9lEMHI8KRZgjIINowYF5ACLyjsXjhbyk81RRpqWU2DSGdDtF1P6STB8E6tuNH82 nIvloVw2lRl6gVqW44z1i0V9n7mQRgFitjxMwtJseAAFioAHhgG/7ESlHwLAArATHV6a PpjDJrhlGYi4viiK7+qqxePKzAU+imAaMr+WKM+woJ2JFI9k/2XFoQtmbcfkfsBVguK/ Ar+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9nuQ1A5wW49KkD24auT2toreqMCyriAoldfaGpol7FQ=; b=BJRtf8LOJFlXaTUHrKCBJ6wrL1rSsv6o4WVt3a9gkosTbDyBV5GBrmXxpGgHf14zLN 58rbJcLfPu+icMry4wyeW/qvsschRGeO4OT1z6zkiW9IAg9cm2KuT7b15u13xQdwbeC0 Vk5MR/Z14rJPBJtTPayAJMnqqWyERQzw/8L5YMVTijun9heKAKsZUGKmmPxHdRcRQTWV CBp/La0tDFgxKOJzK+2H9JIrKSlCSGmsI7xM6DotB7LrbO3eOBXu3xGpmlm6lr14R7pQ oBz7BJytquf/FTs/qxnehvzq2kfnC5RxczbopUVMoGMAd6M5iHyUaK5XVD4xz1pbmDC8 9/eA== X-Gm-Message-State: AIVw111F4x0rg1+rSaTGRp1CxWt0n7ZAKAixhIxXXj2C5/bCuX7faKP9 uL44jN4stRyvArtUs29lu8r3xwT88Q== X-Received: by 10.31.51.70 with SMTP id z67mr314970vkz.16.1501068788783; Wed, 26 Jul 2017 04:33:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.184.2 with HTTP; Wed, 26 Jul 2017 04:32:38 -0700 (PDT) In-Reply-To: References: <2172131.duP9I9jcZS@nico-work> From: Fabian Hueske Date: Wed, 26 Jul 2017 13:32:38 +0200 Message-ID: Subject: =?UTF-8?Q?Re=3A_How_can_I_set_charset_for_flink_sql=EF=BC=9F?= To: Ted Yu Cc: =?UTF-8?B?56iL6aql?= , Nico Kruber , user Content-Type: multipart/alternative; boundary="001a1144a82ef05ea6055536cf3a" archived-at: Wed, 26 Jul 2017 11:33:21 -0000 --001a1144a82ef05ea6055536cf3a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable As Timo proposed, I would implement a Scalar user-defined function which returns a boolean and use that instead of LIKE. Have a look here [1]. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/table/udfs.= html#scalar-functions 2017-07-26 3:47 GMT+02:00 Ted Yu : > Logged CALCITE-1903 for this bug. > > FYI > > On Tue, Jul 25, 2017 at 6:39 PM, =E7=A8=8B=E9=AA=A5 wro= te: > >> OK=EF=BC=8Cthanks for remind me. >> >> My sql like this(contain a Chinese word): >> >> SELECT >> 'HIGH' AS LEVEL, >> 'Firewall uplink bandwidth exception:greater than 10000' AS content, >> `system.process.username`, >> `system.process.memory.rss.bytes` >> FROM >> test >> WHERE >> `system.process.username` LIKE '%=E9=AB=98=E5=8D=B1%' >> AND >> `system.process.memory.rss.bytes` > 10000 >> >> Get exception when I submit the job to cluster. >> >> Caused by: org.apache.calcite.runtime.CalciteException: Failed to encode >> '%=E9=AB=98=E5=8D=B1%' in character set 'ISO-8859-1' >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> ~[na:1.8.0_45] >> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) >> ~[na:1.8.0_45] >> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown >> Source) ~[na:1.8.0_45] >> at java.lang.reflect.Constructor.newInstance(Unknown Source) >> ~[na:1.8.0_45] >> at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.jav= a:463) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.util.NlsString.(NlsString.java:81) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:864) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.rex.RexBuilder.makeCharLiteral(RexBuilder.java:105= 1) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convert >> Literal(SqlNodeToRexConverterImpl.java:117) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRe= lConverter.java:4408) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRe= lConverter.java:3787) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql.SqlLiteral.accept(SqlLiteral.java:427) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.conv >> ertExpression(SqlToRelConverter.java:4321) ~[flink-table_2.11-1.3.1.jar:= 1 >> .3.1] >> at org.apache.calcite.sql2rel.StandardConvertletTable.convertEx >> pressionList(StandardConvertletTable.java:968) >> ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCa >> ll(StandardConvertletTable.java:944) ~[flink-table_2.11-1.3.1.jar:1.3.1] >> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCa >> ll(StandardConvertletTable.java:928) ~[flink-table_2.11-1.3.1.jar:1.3.1] >> ... 50 common frames omitted >> >> Is there anyone tell me how to deal with it,thanks! >> >> ------------------ =E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6 ----------------= -- >> *=E5=8F=91=E4=BB=B6=E4=BA=BA:* "Nico Kruber";; >> *=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4:* 2017=E5=B9=B47=E6=9C=8825=E6=97= =A5(=E6=98=9F=E6=9C=9F=E4=BA=8C) =E6=99=9A=E4=B8=8A11:48 >> *=E6=94=B6=E4=BB=B6=E4=BA=BA:* "user"; >> *=E6=8A=84=E9=80=81:* "=E7=A8=8B=E9=AA=A5"; >> *=E4=B8=BB=E9=A2=98:* Re: How can I set charset for flink sql=EF=BC=9F >> >> Please, for the sake of making your email searchable, do not post stack >> traces >> as screenshots but rather text into your email. >> >> On Tuesday, 25 July 2017 12:18:56 CEST =E7=A8=8B=E9=AA=A5 wrote: >> > My sql like this(contain a Chinese word) >> > >> > Get exception when I submit the job to cluster. >> > >> > >> > >> > Is there anyone tell me how to deal with it,thanks! >> >> >> > --001a1144a82ef05ea6055536cf3a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
As Timo proposed, I would implement a Scalar user-def= ined function which returns a boolean and use that instead of LIKE.

=
Have a look here [1].


2017-0= 7-26 3:47 GMT+02:00 Ted Yu <yuzhihong@gmail.com>:
Logged=C2=A0CALCITE-1903 for thi= s bug.

FYI

On Tue, J= ul 25, 2017 at 6:39 PM, =E7=A8=8B=E9=AA=A5 <ji_cheng@qq.com> w= rote:
OK=EF=BC=8Cthanks for remind me.

=
My sql like this(contain a Chinese word):
=
SELECT
= 'HIGH' AS LEVEL,
'Firewall uplink bandwidth exception:greater than 10000'= ; AS content,
`syste= m.process.username`,
`system.process.memory.rss.bytes`
FROM
test
WHERE
`system.process.username` LIKE '%= =E9=AB=98=E5=8D=B1%'
AND=C2=A0
`syst= em.process.memory.rss.bytes` > 10000

Get exception when I submit the job to cluster.

=
Caused by: org.apache.calcite.runtime.CalciteExceptio= n: Failed to encode '%=E9=AB=98=E5=8D=B1%' in character set 'IS= O-8859-1'
at sun= .reflect.NativeConstructorAccessorImpl.newInstance0(Native Method= ) ~[na:1.8.0_45]
at = sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Sou= rce) ~[na:1.8.0_45]
= at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unkn= own Source) ~[na:1.8.0_45]
= at java.lang.reflect.Constructor.newInstance(Unknown Source) ~[= na:1.8.0_45]
at org.= apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.jav= a:463) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.runtime.Resources$Ex= Inst.ex(Resources.java:572) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite= .util.NlsString.<init>(NlsString.java:81) ~[flink-table_2.11-1.3= .1.jar:1.3.1]
a= t org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:8= 64) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.rex.RexBuilder.makeChar= Literal(RexBuilder.java:1051) ~[flink-table_2.11-1.3.1.jar:1.3.1]=
at org.apache.calci= te.sql2rel.SqlNodeToRexConverterImpl.convertLiteral(SqlNodeToRexC= onverterImpl.java:117) ~[flink-table_2.11-1.3.1.jar:1.3.1]
<= div> at org.apache.calcite.sql2= rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:440= 8) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.SqlToRelConverte= r$Blackboard.visit(SqlToRelConverter.java:3787) ~[flink-table_2.11-1.3= .1.jar:1.3.1]
a= t org.apache.calcite.sql.SqlLiteral.accept(SqlLiteral.java:427) ~= [flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.SqlToRelConverter$Bl= ackboard.convertExpression(SqlToRelConverter.java:4321) ~[flink-t= able_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.StandardConvertletTable.conv= ertExpressionList(StandardConvertletTable.java:968) ~[flink-table= _2.11-1.3.1.jar:1.3.1]
= at org.apache.calcite.sql2rel.StandardConvertletTable.convertC= all(StandardConvertletTable.java:944) ~[flink-table_2.11-1.3.1.ja= r:1.3.1]
at org= .apache.calcite.sql2rel.StandardConvertletTable.convertCall(Stand= ardConvertletTable.java:928) ~[flink-table_2.11-1.3.1.jar:1.3.1]<= /div>
... 50 common frames= omitted

Is there anyone tell me how t= o deal with it,thanks!

------------------=C2=A0= =E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6=C2=A0------------------
=E5=8F=91= =E4=BB=B6=E4=BA=BA:=C2=A0"Nico Kruber";<nico@data-artisans.com&g= t;;
=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4:=C2=A02017=E5=B9= =B47=E6=9C=8825=E6=97=A5(=E6=98=9F=E6=9C=9F=E4=BA=8C) =E6=99=9A=E4=B8=8A11:= 48
=E6=94=B6=E4=BB=B6=E4=BA=BA:=C2=A0"user"<<= a href=3D"mailto:user@flink.apache.org" target=3D"_blank">user@flink.apache= .org>;
=E6=8A=84=E9=80=81:=C2=A0"=E7=A8= =8B=E9=AA=A5"<= ji_cheng@qq.com>;
=E4=B8=BB=E9=A2=98:=C2=A0Re: How= can I set charset for flink sql=EF=BC=9F

Please, for the sake of making your em= ail searchable, do not post stack traces
as screenshots but rather text= into your email.

On Tuesday, 25 July 2017 12:18:56 CEST =E7=A8=8B= =E9=AA=A5 wrote:
> My sql like this(contain a Chinese word)
> <= br>> Get exception when I submit the job to cluster.
>
> >
> Is there anyone tell me how to deal with it,thanks!

=


--001a1144a82ef05ea6055536cf3a--