flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jark Wu <imj...@gmail.com>
Subject Re: FlinkSQL 1.10 DDL无法指定水印
Date Thu, 12 Mar 2020 02:08:03 GMT
Benchao is right.

嵌套字段是无法直接访问的,需要逐级引用到。

On Thu, 12 Mar 2020 at 00:45, Benchao Li <libenchao@gmail.com> wrote:

> Hi 周炎,
>
> 你的`date` 和 `time`都是在嵌套结构内部的字段,需要用*request.`value`.`date`*和*
> request.`value`.`time`*来使用它们。
>
> 周炎 <yarnchou@gmail.com> 于2020年3月11日周三 下午5:42写道:
>
> > DDL语句如下:
> > CREATE TABLE ods_usage_naga_dsp_filter (
> >     request row<`value` row<DSP_FILTER_LOG row<`date` varchar,placementid
> > varchar,reqid varchar,source varchar,`time` varchar,`filter`
> > array<row<filterName varchar,planId varchar>> >>>,
> >     event_ts as to_timestamp(concat(`date`,`time`),'yyyy-MM-ddHH:mm:ss'),
> >     WATERMARK FOR event_ts AS event_ts - interval '60' second
> > )WITH (
> > 'connector.type' = 'kafka',
> > 'format.fail-on-missing-field'='false',
> > 'connector.version' = 'universal',
> > 'connector.topic' = 'xxfilter',
> > 'connector.startup-mode' = 'latest-offset',
> > 'connector.properties.zookeeper.connect'='xx:2181',
> > 'connector.properties.bootstrap.servers'='xx:9092',
> > 'connector.properties.group.id'='xx_test',
> > 'update-mode' = 'append',
> > 'format.type' = 'json'
> > );
> > 启动程序报错:
> > org.apache.flink.client.program.ProgramInvocationException: The main
> method
> > caused an error: From line 3, column 37 to line 3, column 42: Unknown
> > identifier 'date'
> > at
> >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
> > at
> >
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
> > at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
> > at
> >
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
> > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
> > at
> >
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
> > at
> >
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:422)
> > at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> > at
> >
> >
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
> > Caused by: org.apache.calcite.runtime.CalciteContextException: From line
> 3,
> > column 37 to line 3, column 42: Unknown identifier 'date'
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at
> >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> > at
> >
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834)
> > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4841)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5667)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
> > at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:317)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
> > at
> >
> >
> org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
> > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:237)
> > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
> > at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
> > at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:236)
> > at org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:407)
> > at org.apache.calcite.sql.SqlFunction.validateCall(SqlFunction.java:200)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5304)
> > at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:943)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateParameterizedExpression(SqlValidatorImpl.java:930)
> > at
> >
> >
> org.apache.flink.table.planner.operations.SqlToOperationConverter.createTableSchema(SqlToOperationConverter.java:490)
> > at
> >
> >
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convertCreateTable(SqlToOperationConverter.java:177)
> > at
> >
> >
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:130)
> > at
> >
> >
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
> > at
> >
> >
> org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlUpdate(TableEnvironmentImpl.java:484)
> > at
> > com.cootek.streaming.flink.SQLSubmit$.callCreateTable(SQLSubmit.scala:31)
> > at com.cootek.streaming.flink.SQLSubmit$.callCommand(SQLSubmit.scala:15)
> > at
> >
> >
> com.cootek.streaming.FlinkBootstrap$$anonfun$1.apply(FlinkBootstrap.scala:26)
> > at
> >
> >
> com.cootek.streaming.FlinkBootstrap$$anonfun$1.apply(FlinkBootstrap.scala:26)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:891)
> > at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> > at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> > at
> >
> >
> com.cootek.streaming.FlinkBootstrap$.delayedEndpoint$com$cootek$streaming$FlinkBootstrap$1(FlinkBootstrap.scala:26)
> > at
> >
> >
> com.cootek.streaming.FlinkBootstrap$delayedInit$body.apply(FlinkBootstrap.scala:11)
> > at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
> > at
> scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
> > at scala.App$$anonfun$main$1.apply(App.scala:76)
> > at scala.App$$anonfun$main$1.apply(App.scala:76)
> > at scala.collection.immutable.List.foreach(List.scala:392)
> > at
> >
> >
> scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
> > at scala.App$class.main(App.scala:76)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
> > ... 11 more
> > Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Unknown
> > identifier 'date'
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at
> >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> > at
> >
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> > at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
> > ... 63 more
> > flink无法识别关键字date/time??
> >
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message