flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bin Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6115) Need more helpful error message when trying to serialize a tuple with a null field
Date Sat, 23 Sep 2017 08:14:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177627#comment-16177627
] 

Bin Wang commented on FLINK-6115:
---------------------------------

I am a new user of Flink, I would like to recommend to support null serializing in Tuple.
This is what I am doing:
I am working on calculating some of the properties of a data instance. Before calculating,
the property is null. I load the data from Cassandra DB. Firstly I need to filter which data
hasn't been calculated. As you know, A null property cannot be filtered in Cassandra 2.x without
secondary index. Than I am facing this problem. I want to use filter chain to find the data
with null property, but filter is not work with data having property. :(

> Need more helpful error message when trying to serialize a tuple with a null field
> ----------------------------------------------------------------------------------
>
>                 Key: FLINK-6115
>                 URL: https://issues.apache.org/jira/browse/FLINK-6115
>             Project: Flink
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.0
>            Reporter: Luke Hutchison
>
> When Flink tries to serialize a tuple with a null field, you get the following, which
has no information about where in the program the problem occurred (all the stack trace lines
are in Flink, not in user code).
> {noformat}
> Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job
execution failed.
> 	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply$mcV$sp(JobManager.scala:900)
> 	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:843)
> 	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:843)
> 	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
> 	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
> 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
> 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
> 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.lang.IllegalArgumentException: The record must not be null.
> 	at org.apache.flink.api.common.typeutils.base.array.StringArraySerializer.serialize(StringArraySerializer.java:73)
> 	at org.apache.flink.api.common.typeutils.base.array.StringArraySerializer.serialize(StringArraySerializer.java:33)
> 	at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:124)
> 	at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
> 	at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:56)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:77)
> 	at org.apache.flink.runtime.io.network.api.writer.RecordWriter.sendToTarget(RecordWriter.java:113)
> 	at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:88)
> 	at org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
> 	at org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
> 	at org.apache.flink.runtime.operators.chaining.ChainedMapDriver.collect(ChainedMapDriver.java:79)
> 	at org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
> 	at org.apache.flink.api.java.operators.translation.PlanFilterOperator$FlatMapFilter.flatMap(PlanFilterOperator.java:51)
> 	at org.apache.flink.runtime.operators.FlatMapDriver.run(FlatMapDriver.java:108)
> 	at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:490)
> 	at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:355)
> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:655)
> 	at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The only thing I can tell from this is that it happened somewhere in a flatMap (but I
have dozens of them in my code). Surely there's a way to pull out the source file name and
line number from the program DAG node when errors like this occur?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message