flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuan,Youjun" <yuanyou...@baidu.com>
Subject 答复: Conversion to relational algebra failed to preserve datatypes
Date Fri, 14 Sep 2018 14:55:35 GMT
Hi Timo,

I believe it has something to do with the number of field in the source. I can reproduce the
issue with very simple SQL, like:
INSERT INTO IVC_ALARM_PUBLIC_OUT SELECT rowtime as ts FROM IVC_ALARM_PUBLIC_IN
Where rowtime actually represent proctime (I registered it as rowtime.proctime).

If I remove a few fields in the source, then everything goes fine.

Thanks
Youjun

发件人: Timo Walther <twalthr@apache.org>
发送时间: Friday, September 14, 2018 9:17 PM
收件人: user@flink.apache.org
主题: Re: Conversion to relational algebra failed to preserve datatypes

Hi,

could you maybe post the query that caused the exception? I guess the exception is related
to a time attribute [1] for the optimizer time attributes and timestamps make no difference
however they have a slightly different data type that might have caused the error. I think
is a bug that should be fixed, once we have more context.

Regards,
Timo


[1] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming.html#time-attributes


Am 14.09.18 um 10:49 schrieb Yuan,Youjun:
Hi,

I am getting the following error while submitting job to a cluster, which seems failed to
compare 2 RelDateTypes, though they seems identical (from the error message), and everything
is OK if I run it locally.
I guess calcite failed to compare the first field named ts, of type TIMESTAMP(3), because:

  *   If I don’t select ts, then everything goes fine
  *   If I cast ts to other type, like SELECT cast(ts AS TIMESTAMP), then everything is fine
  *   If I switch to EventTime, the issue also goes away. Currently it’s ProcessTime

I am using Flink 1.4, and submitting job to a standalone cluster.

Below are the error:
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Caused by: org.apache.flink.client.program.ProgramInvocationException: The program caused
an error:
at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:93)
at org.apache.flink.client.program.ClusterClient.getOptimizedPlan(ClusterClient.java:334)
at org.apache.flink.runtime.webmonitor.handlers.JarActionHandler.getJobGraphAndClassLoader(JarActionHandler.java:76)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleJsonRequest$0(JarRunHandler.java:69)
... 9 more
Caused by: java.lang.AssertionError: Conversion to relational algebra failed to preserve datatypes:
validated type:
RecordType(TIMESTAMP(3) NOT NULL ts, VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
userId, VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" batchId,
VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" projectId, VARCHAR(65536)
CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" vehicleId, CHAR(5) CHARACTER SET
"UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL field0, VARCHAR(65536) CHARACTER SET
"UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" rule0, DOUBLE threshold0, DOUBLE field_value0,
VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" alarmId) NOT NULL
converted type:
RecordType(TIMESTAMP(3) NOT NULL ts, VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
userId, VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" batchId,
VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" projectId, VARCHAR(65536)
CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" vehicleId, CHAR(5) CHARACTER SET
"UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL field0, VARCHAR(65536) CHARACTER SET
"UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" rule0, DOUBLE threshold0, DOUBLE field_value0,
VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" alarmId) NOT NULL
...
at org.apache.calcite.sql2rel.SqlToRelConverter.checkConvertedType(SqlToRelConverter.java:451)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:567)
at org.apache.flink.table.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:106)
----------------------------------------------------------------------------------------------------------------------------------------------------------------

thanks in advance,
youjun


Mime
View raw message