flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yun Tang <myas...@live.com>
Subject Re: flink遇到 valueState 自身的 NPE
Date Thu, 09 Jan 2020 19:05:28 GMT
Hi Kevin

帮你看了下代码,问题定位到了。
容易发生的原因在于你的可见性配置成了 ReturnExpiredIfNotCleanedUp,而Flink会先发现数据expire,然后触发clear操作,之后再返回expire的数据 [1]。建议修改一下第39行的判断条件,每次取数据时都做是否为null的判断。


[1] https://github.com/apache/flink/blob/dba5b9e0138b667c3ecd32f7b16645d531477720/flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/AbstractTtlDecorator.java#L96

祝好
唐云

________________________________
From: Kevin Liao <liaoxu@gmail.com>
Sent: Friday, January 10, 2020 1:08
To: Yun Tang <myasuka@live.com>
Cc: user-zh@flink.apache.org <user-zh@flink.apache.org>
Subject: Re: flink遇到 valueState 自身的 NPE

谢答,首先贴的代码确实是运行的程序

此外刚刚又通过打印 log 确认了 uniqMark == null 是 false

我现在的怀疑点是这个地方

if (null != uniqMark.value() && uniqMark.value()) {

第一处uniqMark.value()可以取到结果(不为 null),同时由于 ttl 策略会触发
clear,进而导致第二个uniqMark.value()取出来的就是 null
了(看来编译器没有对这种写法做优化,就是真实的执行了两次value()函数),追了下代码似乎可以支持我的猜测,所以准备明天验证下这个猜测

明天有消息同步,谢谢

Yun Tang <myasuka@live.com> 于2020年1月10日周五 上午12:59写道:

> Hi Kevin
>
> State TTL 是清理的state中的数据条目entry,不是清理state在map函数中的对象本身。所以无论如何,作为value
> state对象的uniqMark 是不会因为TTL而变成null的。
>
>
> 我注意到你的作业即使发生failover之后,立刻恢复的时候,仍然遇到了这个NPE问题,我怀疑你实际运行的代码第39行并不是你贴出来的代码,很有可能是对应你的代码的第34行,也就是map方法的输入RDLog是null,这也符合作业成功restore之后,又再次立即遇到failover的场景,也就是处理到了非法“脏数据”,导致作业不断failover。建议你按照这个思路确认排除一下。
>
> 祝好
> 唐云
> ------------------------------
> *From:* Kevin Liao <liaoxu@gmail.com>
> *Sent:* Thursday, January 9, 2020 23:17
> *To:* user-zh@flink.apache.org <user-zh@flink.apache.org>
> *Subject:* Re: flink遇到 valueState 自身的 NPE
>
> 谢答
> 但还有一问题想请教:
>
> 当 valueState 触发 ttl 被回收后,这里的引用应该也会被 gc 掉,就会变成 null 了吧?难道是 operator 原来处理这个
> key 的线程会被一并回收掉,下次这个 key 再来时其实已经是另外新生成的线程提供服务了(这样肯定要重新调用一次 open
> 方法)?看了代码但还没搞明白,求解惑,谢谢
>
> Benchao Li <libenchao@gmail.com> 于2020年1月9日周四 下午8:59写道:
>
> > 我感觉这个地方好像没有道理会有`uniqMark`变成`null`,
> > 除非是什么地方反序列化出来`StreamMap`,并且没有调用`StreamMap.open()`.
> > 但是看起来`StreamTask`是可以保证先调用`open`,再调用operator的处理函数的。我也看不出来这个地方有什么问题。
> >
> > Kevin Liao <liaoxu@gmail.com> 于2020年1月9日周四 下午8:15写道:
> >
> > > https://tva4.sinaimg.cn/large/63137227ly1gaqkn1nlykj20mm0wvgq8.jpg
> > >
> > > 抱歉,再试试这个
> > >
> > > Benchao Li <libenchao@gmail.com> 于2020年1月9日周四 下午8:13写道:
> > >
> > > > 我这边点开是 403 Forbidden
> > > >
> > > > Kevin Liao <liaoxu@gmail.com> 于2020年1月9日周四 下午8:09写道:
> > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://gm1.ggpht.com/FZGtbLggyPPZ_BoU0gt2SQTv7fyhNOKu03ZjsKq7G6DtqWJ5DY0NmL-2s64P-LUzedbTm8DE_FeggNtPAb4VEmypAoPfW8VFSFxOWxMGBvMi5G6xHoZ3THKPYHnAj8KydQ02OjvV-R3IEwBvwIDnZRmwCv3ohyjPF76gbvOOYrKzgaLb_pykWsQDpvROHr3lgU2rezH33Jt3xJEOjXGjHsUFUxiil0PYkQFdA0BP77lypYQLw4RL8BxMz3HfaCiNAGb_q5w8JNmckHLU3g9EuPgtqj6WP3XDv07PBuCXMvmfNcFbAciMeJuOOeE8VBqDCacjuiDtJzVrK1boxcBnzFvT_QazOwaJ27SSuJ_u5KCerTURen2vLBF1RN-x9eOVz9wg6w1oXyMAF7LMjGsYsVzUu3It5AyzLkm-_znosNtAJp2AW_qGmGo-k02fcrMjUoELiGvqn6W1kScnFI4gNWi_dpZe0Uoq1zF2m1crww1oNGOeRjFlCK_-iC19CPfsTVCtwN3tdKnaKdLe2TbfVdFA0DnBUz8NrhV-mvmZlEwi9-ngK-WOy8yjA4fin1zaE2SJCf2zfBSZwGR2eY_E_WZQiFRmSBI2A7vpoyFvTV3E99MIi0MC5PUAeRiu4v4JIVDkV_yUGIUvoa7pxdf7LpZN_DbikQVk7yES8kxxL5qG2Eae8vftWJuBVi5mWTxvElBgInyUntobXHdxfb2YR4JdBgVPN5QionJiIc9g5i0ClGECZbyHPbsQy4pEVw=s0-l75-ft-l75-ft
> > > > >
> > > > > 谢谢,看看能否看见
> > > > >
> > > > > Benchao Li <libenchao@gmail.com> 于2020年1月9日周四 下午8:07写道:
> > > > >
> > > > > > hi Kevin,
> > > > > >
> > > > > > 邮件里面贴不了图片,如果要贴图片,需要用一些第三方的图床工具。
> > > > > > 或者你可以直接贴文字?
> > > > > >
> > > > > > Kevin Liao <liaoxu@gmail.com> 于2020年1月9日周四 下午7:10写道:
> > > > > >
> > > > > > > [image: B40C260D-DCC3-4B7D-A024-3839803C2234.png]
> > > > > > >
> > > > > > > Benchao Li <libenchao@gmail.com> 于2020年1月9日周四 下午6:42写道:
> > > > > > >
> > > > > > >> hi Kevin,
> > > > > > >>
> > > > > > >> 能贴一下MyMapFunction2.java:39 这里的代码吗?
> 从上面的日志看不出来是valueState是null呢。
> > > > > > >>
> > > > > > >> Kevin Liao <liaoxu@gmail.com> 于2020年1月9日周四 下午5:57写道:
> > > > > > >>
> > > > > > >> > 早上发现任务异常,task 在不停重启,遂查看 jm 日志,最开始的报错是这样的
> > > > > > >> >
> > > > > > >> > ```
> > > > > > >> > 2020-01-09 05:14:04.087
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (3/6) (d0e6c4a05d0274c18a4a3df41ab5ff1b)
> > switched
> > > > from
> > > > > > >> > RUNNING to FAILED.
> > > > > > >> > java.lang.NullPointerException: null
> > > > > > >> > at
> com.sogou.qidian.MyMapFunction2.map(MyMapFunction2.java:39)
> > > > > > >> > at
> com.sogou.qidian.MyMapFunction2.map(MyMapFunction2.java:25)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
> > > > > > >> > at
> > > > > > >> > org.apache.flink.streaming.runtime.io
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > .StreamOneInputProcessor.processElement(StreamOneInputProcessor.java:164)
> > > > > > >> > at
> > > > > > >> > org.apache.flink.streaming.runtime.io
> > > > > > >> >
> > > > > >
> > > .StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:143)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406)
> > > > > > >> > at
> > > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
> > > > > > >> > at
> > org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
> > > > > > >> > at java.lang.Thread.run(Thread.java:748)
> > > > > > >> > 2020-01-09 05:14:04.088
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  o.a.f.r.e.failover.flip1.RestartPipelinedRegionStrategy  -
> > > > > > Calculating
> > > > > > >> > tasks to restart to recover the failed task
> > > > > > >> > 90bea66de1c231edf33913ecd54406c1_2.
> > > > > > >> > 2020-01-09 05:14:04.088
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  o.a.f.r.e.failover.flip1.RestartPipelinedRegionStrategy  -
> 12
> > > > tasks
> > > > > > >> should
> > > > > > >> > be restarted to recover the failed task
> > > > > > >> 90bea66de1c231edf33913ecd54406c1_2.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (6/6) (ac52050e60236cd1efcd262c8240cd25)
> > > switched
> > > > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (5/6) (cf4ff0c189315b27e7e2178d8c60e49f)
> > > switched
> > > > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (4/6) (8c8b07cb39a3f682f41f102e614765e6)
> > > switched
> > > > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (3/6) (34aadddbffe9f61b1916bcd1427ced96)
> > > switched
> > > > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (6/6) (de624cf5c9d4dec6fe68d4800c701457)
> > switched
> > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (4/6) (e52c1e70884a6599205f9e0f5b092bc0)
> > switched
> > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (5/6) (60496dddb4bc885ee37a6025662080ad)
> > switched
> > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.089
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (2/6) (bf8515b4f9e852182a5519102fe4fdf3)
> > switched
> > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.090
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (2/6) (bb14d5776c53babcc57edd65bf7159b0)
> > > switched
> > > > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.090
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (1/6) (4c7cd6eaf5c3ca9c2b0db73e7d230a9e)
> > > switched
> > > > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.090
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (1/6) (4a157d98db2f2efad72035af279433ff)
> > switched
> > > > from
> > > > > > >> > RUNNING to CANCELING.
> > > > > > >> > 2020-01-09 05:14:04.096
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (1/6) (4c7cd6eaf5c3ca9c2b0db73e7d230a9e)
> > > switched
> > > > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.101
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (2/6) (bb14d5776c53babcc57edd65bf7159b0)
> > > switched
> > > > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.103
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (5/6) (cf4ff0c189315b27e7e2178d8c60e49f)
> > > switched
> > > > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.115
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (2/6) (bf8515b4f9e852182a5519102fe4fdf3)
> > switched
> > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.119
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (4/6) (e52c1e70884a6599205f9e0f5b092bc0)
> > switched
> > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.140
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (6/6) (de624cf5c9d4dec6fe68d4800c701457)
> > switched
> > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.140
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > > Discarding
> > > > > > >> the
> > > > > > >> > results produced by task execution
> > > > de624cf5c9d4dec6fe68d4800c701457.
> > > > > > >> > 2020-01-09 05:14:04.141
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (6/6) (ac52050e60236cd1efcd262c8240cd25)
> > > switched
> > > > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.141
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (5/6) (60496dddb4bc885ee37a6025662080ad)
> > switched
> > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.141
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > > Discarding
> > > > > > >> the
> > > > > > >> > results produced by task execution
> > > > 60496dddb4bc885ee37a6025662080ad.
> > > > > > >> > 2020-01-09 05:14:04.142
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (4/6) (8c8b07cb39a3f682f41f102e614765e6)
> > > switched
> > > > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.142
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (1/6) (4a157d98db2f2efad72035af279433ff)
> > switched
> > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.142
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > > Discarding
> > > > > > >> the
> > > > > > >> > results produced by task execution
> > > > 4a157d98db2f2efad72035af279433ff.
> > > > > > >> > 2020-01-09 05:14:04.143
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (3/6) (34aadddbffe9f61b1916bcd1427ced96)
> > > switched
> > > > > > from
> > > > > > >> > CANCELING to CANCELED.
> > > > > > >> > 2020-01-09 05:14:04.143
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >
> o.a.f.r.e.failover.AdaptedRestartPipelinedRegionStrategyNG  -
> > > > > Finally
> > > > > > >> > restart 12 tasks to recover from task failure.
> > > > > > >> > 2020-01-09 05:14:04.144
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> -
> > > > > > Restoring
> > > > > > >> job
> > > > > > >> > 7907d93e17fccf460069c2ca7f13d7e7 from latest valid
> checkpoint:
> > > > > > >> Checkpoint
> > > > > > >> > 720 @ 1578518011978 for 7907d93e17fccf460069c2ca7f13d7e7.
> > > > > > >> > 2020-01-09 05:14:04.145
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> -
> > No
> > > > > > master
> > > > > > >> > state to restore
> > > > > > >> > 2020-01-09 05:14:04.145
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (2/6) (d83eee0b90a0e4a4dcff3e1ca6bed8b6)
> > > switched
> > > > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.145
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (4/6) (da6e91a7475712da5ca52fc82069345f)
> > > switched
> > > > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.145
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (1/6) (ffa3b56b4e1321aa5ec94e040921e223)
> > > switched
> > > > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.145
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (3/6) (5c3fb959f79d53cdca506188ed40cc6d)
> > > switched
> > > > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (5/6) (7a2252b0a18cd91211adae3ce3b613be)
> > > switched
> > > > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (6/6) (6010486f42fc1e0e7c93623174bf0477)
> > > switched
> > > > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (1/6) (eb06b4212d17f10cf4a8bcc812eaa073)
> > switched
> > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (2/6) (af4e3fc75729a2d035db42ec95e44189)
> > switched
> > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (3/6) (adb9b0aabe0234fe336211f5e44e18f3)
> > switched
> > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (4/6) (9ad2c9ee62ff804683284b0cdf571c99)
> > switched
> > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.146
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (6/6) (1e66dc5791940cf7e213b1b3cf8cf1fd)
> > switched
> > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (5/6) (4cb656b991141acbbe9e1033103ab470)
> > switched
> > > > from
> > > > > > >> > CREATED to SCHEDULED.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (2/6) (d83eee0b90a0e4a4dcff3e1ca6bed8b6)
> > > switched
> > > > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> > Source: Custom Source -> Filter (2/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000003 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=15754)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (4/6) (da6e91a7475712da5ca52fc82069345f)
> > > switched
> > > > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> > Source: Custom Source -> Filter (4/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000002 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=20658)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (1/6) (ffa3b56b4e1321aa5ec94e040921e223)
> > > switched
> > > > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> > Source: Custom Source -> Filter (1/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000003 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=15754)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (3/6) (5c3fb959f79d53cdca506188ed40cc6d)
> > > switched
> > > > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> > Source: Custom Source -> Filter (3/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000002 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=20658)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (5/6) (7a2252b0a18cd91211adae3ce3b613be)
> > > switched
> > > > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> > Source: Custom Source -> Filter (5/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000003 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=15754)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (6/6) (6010486f42fc1e0e7c93623174bf0477)
> > > switched
> > > > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> > Source: Custom Source -> Filter (6/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000002 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=20658)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (1/6) (eb06b4212d17f10cf4a8bcc812eaa073)
> > switched
> > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> Map ->
> > > > > > >> > Filter -> Sink: Unnamed (1/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000002 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=20658)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (2/6) (af4e3fc75729a2d035db42ec95e44189)
> > switched
> > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> Map ->
> > > > > > >> > Filter -> Sink: Unnamed (2/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000003 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=15754)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (3/6) (adb9b0aabe0234fe336211f5e44e18f3)
> > switched
> > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> Map ->
> > > > > > >> > Filter -> Sink: Unnamed (3/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000003 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=15754)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (4/6) (9ad2c9ee62ff804683284b0cdf571c99)
> > switched
> > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> Map ->
> > > > > > >> > Filter -> Sink: Unnamed (4/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000003 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=15754)
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (6/6) (1e66dc5791940cf7e213b1b3cf8cf1fd)
> > switched
> > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.147
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> Map ->
> > > > > > >> > Filter -> Sink: Unnamed (6/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000002 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=20658)
> > > > > > >> > 2020-01-09 05:14:04.148
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (5/6) (4cb656b991141acbbe9e1033103ab470)
> > switched
> > > > from
> > > > > > >> > SCHEDULED to DEPLOYING.
> > > > > > >> > 2020-01-09 05:14:04.148
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > > Deploying
> > > > > > >> Map ->
> > > > > > >> > Filter -> Sink: Unnamed (5/6) (attempt #2) to
> > > > > > >> > container_e06_1577354972821_672000_01_000002 @
> > > > > > >> > rsync.slave713.mars.hadoop.js.ted (dataPort=20658)
> > > > > > >> > 2020-01-09 05:14:04.159
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (1/6) (ffa3b56b4e1321aa5ec94e040921e223)
> > > switched
> > > > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.160
> > [flink-akka.actor.default-dispatcher-28]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (2/6) (d83eee0b90a0e4a4dcff3e1ca6bed8b6)
> > > switched
> > > > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.161
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (5/6) (7a2252b0a18cd91211adae3ce3b613be)
> > > switched
> > > > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.168
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (4/6) (9ad2c9ee62ff804683284b0cdf571c99)
> > switched
> > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.169
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (4/6) (da6e91a7475712da5ca52fc82069345f)
> > > switched
> > > > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.170
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (6/6) (6010486f42fc1e0e7c93623174bf0477)
> > > switched
> > > > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.170
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (2/6) (af4e3fc75729a2d035db42ec95e44189)
> > switched
> > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.173
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (3/6) (adb9b0aabe0234fe336211f5e44e18f3)
> > switched
> > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.210
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> > > Source:
> > > > > > >> Custom
> > > > > > >> > Source -> Filter (3/6) (5c3fb959f79d53cdca506188ed40cc6d)
> > > switched
> > > > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.230
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (1/6) (eb06b4212d17f10cf4a8bcc812eaa073)
> > switched
> > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.231
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (6/6) (1e66dc5791940cf7e213b1b3cf8cf1fd)
> > switched
> > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.245
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (5/6) (4cb656b991141acbbe9e1033103ab470)
> > switched
> > > > from
> > > > > > >> > DEPLOYING to RUNNING.
> > > > > > >> > 2020-01-09 05:14:04.922
> > [flink-akka.actor.default-dispatcher-14]
> > > > > INFO
> > > > > > >> >  org.apache.flink.runtime.executiongraph.ExecutionGraph  -
> Map
> > > ->
> > > > > > >> Filter ->
> > > > > > >> > Sink: Unnamed (3/6) (adb9b0aabe0234fe336211f5e44e18f3)
> > switched
> > > > from
> > > > > > >> > RUNNING to FAILED.
> > > > > > >> > java.lang.NullPointerException: null
> > > > > > >> > at
> com.sogou.qidian.MyMapFunction2.map(MyMapFunction2.java:39)
> > > > > > >> > at
> com.sogou.qidian.MyMapFunction2.map(MyMapFunction2.java:25)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
> > > > > > >> > at
> > > > > > >> > org.apache.flink.streaming.runtime.io
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > >
> > .StreamOneInputProcessor.processElement(StreamOneInputProcessor.java:164)
> > > > > > >> > at
> > > > > > >> > org.apache.flink.streaming.runtime.io
> > > > > > >> >
> > > > > >
> > > .StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:143)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301)
> > > > > > >> > at
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406)
> > > > > > >> > at
> > > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
> > > > > > >> > at
> > org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
> > > > > > >> > at java.lang.Thread.run(Thread.java:748)
> > > > > > >> > ```
> > > > > > >> >
> > > > > > >> > 之后就是不停的循环,怪异的是 MyMapFunction2.java:39内容是
> > > > > > >> > ```
> > > > > > >> >
> > > > > > >> > if (null != uniqMark.value() && uniqMark.value()) {
> > > > > > >> >
> > > > > > >> > ```
> > > > > > >> >
> > > > > > >> > 而报错的时间点距离任务启动差不多就是 12 小时,符合我设置的 valueState 的 ttl
> > > > > > >> >
> > > > > > >> > ```
> > > > > > >> >
> > > > > > >> > @Override
> > > > > > >> > public void open(Configuration parameters) throws Exception
> {
> > > > > > >> >   ValueStateDescriptor<Boolean> descriptor = new
> > > > > > >> > ValueStateDescriptor<>("uniqMark",
> > > > > > >> >       TypeInformation.of(Boolean.class));
> > > > > > >> >   StateTtlConfig ttlConfig =
> > > > > StateTtlConfig.newBuilder(Time.hours(1))
> > > > > > >> >       .setUpdateType(UpdateType.OnCreateAndWrite)
> > > > > > >> >
> > > > >  .setStateVisibility(StateVisibility.ReturnExpiredIfNotCleanedUp)
> > > > > > >> >       .cleanupIncrementally(1000, false)
> > > > > > >> >       .build();
> > > > > > >> >   descriptor.enableTimeToLive(ttlConfig);
> > > > > > >> >   uniqMark = getRuntimeContext().getState(descriptor);
> > > > > > >> > }
> > > > > > >> >
> > > > > > >> > ```
> > > > > > >> >
> > > > > > >> > 所以我猜测在什么极端情况下会出现 valueState 自身变为 null 的情况么?(比如刚刚被清理的瞬间)?
> > > > > > >> >
> > > > > > >> > 求各位指正
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >> Benchao Li
> > > > > > >> School of Electronics Engineering and Computer Science, Peking
> > > > > > University
> > > > > > >> Tel:+86-15650713730
> > > > > > >> Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Benchao Li
> > > > > > School of Electronics Engineering and Computer Science, Peking
> > > > University
> > > > > > Tel:+86-15650713730
> > > > > > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Benchao Li
> > > > School of Electronics Engineering and Computer Science, Peking
> > University
> > > > Tel:+86-15650713730
> > > > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > > >
> > >
> >
> >
> > --
> >
> > Benchao Li
> > School of Electronics Engineering and Computer Science, Peking University
> > Tel:+86-15650713730
> > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> >
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message