Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9245318058 for ; Sun, 16 Aug 2015 14:38:15 +0000 (UTC) Received: (qmail 68831 invoked by uid 500); 16 Aug 2015 14:38:15 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 68754 invoked by uid 500); 16 Aug 2015 14:38:15 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 68744 invoked by uid 99); 16 Aug 2015 14:38:15 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 16 Aug 2015 14:38:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id F257318210F for ; Sun, 16 Aug 2015 14:38:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id wkIiKA_PGGC2 for ; Sun, 16 Aug 2015 14:38:13 +0000 (UTC) Received: from mail-lb0-f173.google.com (mail-lb0-f173.google.com [209.85.217.173]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id E70FA20C31 for ; Sun, 16 Aug 2015 14:38:12 +0000 (UTC) Received: by lbcbn3 with SMTP id bn3so68064393lbc.2 for ; Sun, 16 Aug 2015 07:38:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=PCVAd5ajyYAqEuIMHN+dlMOMDqLbdUr+AKwsWVeUS9U=; b=H1KKb68zfOs864VvqKFeTcHPYbwgn5D0z1BsQLA3oSEkAHmex5K0QrPybkQnEsp55P kZwAFjamxkOfLYA3OeEuiEaRsGlBwo/BGTAsTI5vg88c+4W/zdLNQaBN5Dq/7H4o1xWV 0C+Mz4V8gRCiFUT820rpij+V5c8/Ui+Qd/Zw0N4d9iLffNw5ee4wez2s9lWhqxL8RAlt rcXuC28m4Yg5vj7vdZ13XRdPCu6PHkWTHSRP3Uaf6KyCmnHCHJFoeBp4U3J8NKwElvbi Ux04M7OFq9W7J0yNE2sXjVpOPdXvDS9Ru8x1qVl/smJGtDuiKEMi1vhpnxYoHlEOjK/a /kWQ== MIME-Version: 1.0 X-Received: by 10.152.2.41 with SMTP id 9mr52444912lar.65.1439735892044; Sun, 16 Aug 2015 07:38:12 -0700 (PDT) Received: by 10.114.13.69 with HTTP; Sun, 16 Aug 2015 07:38:12 -0700 (PDT) Date: Sun, 16 Aug 2015 10:38:12 -0400 Message-ID: Subject: Serialization and kryo From: jay vyas To: user@flink.apache.org Content-Type: multipart/alternative; boundary=089e013c62586a7196051d6ea2e9 --089e013c62586a7196051d6ea2e9 Content-Type: text/plain; charset=UTF-8 Hi flink. I just ran into the following serialization error in BigPetStore Flink. It appears to be that someone is trying to add elements to a map during serialization. I doubt this is a bug in flink because it would have surely been caught early on. Some possible explanations ... 1 - Probably its a conflict caused by my application's dependencies ? 2 - Or maybe just something about the way the mini-cluster sets up its dependencies?. 3 - Or... Is this related to a guava transitive dependency conflict? DETAILS Our data generator library has these dependencies... Just FYI... [INFO] \- com.github.rnowling.bigpetstore:bigpetstore-data-generator:jar:0.2.1:compile [INFO] +- com.google.guava:guava:jar:18.0:compile [INFO] \- com.google.code.gson:gson:jar:2.3:compile And, heres the exception: Caused by: com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException Serialization trace: fields (com.github.rnowling.bps.datagenerator.datamodels.Product) products (com.github.rnowling.bps.datagenerator.datamodels.Transaction) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116) at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22) at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) Caused by: java.lang.UnsupportedOperationException at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:326) at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:144) at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21) at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) ... 18 more --089e013c62586a7196051d6ea2e9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi flink.

I just ran in= to the following serialization error in BigPetStore Flink. =C2=A0 It appear= s to be that someone is trying to add elements to a map during serializatio= n. =C2=A0 I doubt this is a bug in flink because it would have surely been = caught early on. =C2=A0

Some possible explanations= ...

1 - Probably its a conflict caused by my appl= ication's dependencies ? =C2=A0

2 - Or maybe j= ust something about the way the mini-cluster sets up its dependencies?.

3 - Or... Is this related to a guava transitive depen= dency conflict?

DETAILS

O= ur data generator library has these dependencies... Just FYI...
<= div>[INFO] \- com.github.rnowling.bigpetstore:bigpetstore-data-generator:ja= r:0.2.1:compile
[INFO] =C2=A0 =C2=A0+- com.google.guava:guava:jar= :18.0:compile
[INFO] =C2=A0 =C2=A0\- com.google.code.gson:gson:ja= r:2.3:compile


And, heres the = exception:=C2=A0

Caused by: com.esotericsoftware.k= ryo.KryoException: java.lang.UnsupportedOperationException
Serial= ization trace:
fields (com.github.rnowling.bps.datagenerator.data= models.Product)
products (com.github.rnowling.bps.datagenerator.d= atamodels.Transaction)
at com.esotericsoftware.kryo.serializers.ObjectField= .read(ObjectField.java:125)
at com.esotericsoftware.kryo.serializers.Field= Serializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo= .readClassAndObject(Kryo.java:761)
at com.esotericsoftware.kryo.serializers= .CollectionSerializer.read(CollectionSerializer.java:116)
at com.esotericso= ftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java= :22)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
= at com.eso= tericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
=
at co= m.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.ja= va:528)
= at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)=

Caused by: java.lang.UnsupportedOperat= ionException
at com.google.common.collect.ImmutableMap.put(ImmutableMap.jav= a:326)
= at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeria= lizer.java:144)
at com.esotericsoftware.kryo.serializers.MapSerializer.read= (MapSerializer.java:21)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.j= ava:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectFi= eld.java:106)
... 18 more
--089e013c62586a7196051d6ea2e9--