flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pushpendra Jaiswal (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-4331) Flink is not able to serialize scala classes / Task Not Serializable
Date Mon, 08 Aug 2016 14:41:20 GMT
Pushpendra Jaiswal created FLINK-4331:
-----------------------------------------

             Summary: Flink is not able to serialize scala classes / Task Not Serializable
                 Key: FLINK-4331
                 URL: https://issues.apache.org/jira/browse/FLINK-4331
             Project: Flink
          Issue Type: Bug
          Components: DataStream API
    Affects Versions: 1.1.0
            Reporter: Pushpendra Jaiswal


I have scala class having 2 fields which are vals but flink is saying it doesn't have setters.
Thus task is not serializable.

I tried setters using var but then it says duplicate setter. vals are public then why it is
asking for setters. Flink version 1.1.0

class Impression(val map: Map[String, String],val keySet:Set[String])
==========================================================================

  val preAggregate = stream
.filter(impression => {
    true
})
 .map(impression => {
  val xmap = impression.map
  val values = valFunction(xmap)
  new ImpressionRecord(impression, values._1, values._2, values._3)
})
class Impression does not contain a setter for field map 19:54:49.995 [main] INFO o.a.f.a.java.typeutils.TypeExtractor
- class Impression is not a valid POJO type 19:54:49.997 [main] DEBUG o.a.flink.api.scala.ClosureCleaner$
- accessedFields: Map(class -> Set()) Exception in thread "main" org.apache.flink.api.common.InvalidProgramException:
Task not serializable at org.apache.flink.api.scala.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:172)
at ) Caused by: java.io.NotSerializableException: org.apache.flink.streaming.api.scala.DataStream
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:301) at
org.apache.flink.api.scala.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:170) ...
18 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message