flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jark Wu" <wuchong...@alibaba-inc.com>
Subject Re: HashMap/HashSet Serialization Issue
Date Sat, 07 Jan 2017 11:42:53 GMT
Hi Charith,

The LOG message has nothing to worry about. The HashMap and HashSet members in your POJO is
not a PojoType, 
so they will be handled as GenericType, i.e. will be serialized/deserialized by Kryo. But
your custom POJO type will still 
recognized as PojoType, i.e. will be serialized/deserialized by Flink serialization framework.

In a word, your custom POJO can be serialized/deserialized correctly. There is no issue in
it.  :)

- Jark Wu 

> 在 2017年1月7日,下午12:02,Charith Wickramarachchi <charith.dhanushka@gmail.com>
写道:
> 
> Thanks very much. 
> 
> Regards,
> Charith
> 
> On Fri, Jan 6, 2017 at 7:44 PM, Chen Qin <qinnchen@gmail.com <mailto:qinnchen@gmail.com>>
wrote:
> My understanding is HashMap doesn't work with Flink Native serialization framework, though
I might be wrong.
> 
> This might worth reading
> ​https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization
<https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization>
> 
> -Chen​
> 
> On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi <charith.dhanushka@gmail.com
<mailto:charith.dhanushka@gmail.com>> wrote:
> Hi All, 
> 
> I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I
am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing
the job I am getting following Log message. 
> 
> 17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - No
fields detected for class java.util.HashSet. Cannot be used as a PojoType. Will be handled
as GenericType
> 17:50:53,583 INFO  org.apache.flink.api.java.typeutils.​​TypeExtractor          
  - class java.util.HashMap is not a valid POJO type
> 
> Is there a way to resolve this issue? 
> 
> Thanks,
> Charith
> 
> 
> -- 
> Charith Dhanushka Wickramaarachchi
> 
> Tel  +1 213 447 4253 <>
> Blog  http://charith.wickramaarachchi.org/ <http://charithwiki.blogspot.com/>
> Twitter  @charithwiki <https://twitter.com/charithwiki>
> 
> This communication may contain privileged or other confidential information and is intended
exclusively for the addressee/s. If you are not the intended recipient/s, or believe that
you may have
> received this communication in error, please reply to the sender indicating that fact
and delete the copy you received and in addition, you should not print, copy, retransmit,
disseminate, or otherwise use the information contained in this communication. Internet communications
cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept
liability for any errors or omissions
> 
> 
> 
> 
> -- 
> Charith Dhanushka Wickramaarachchi
> 
> Tel  +1 213 447 4253 <>
> Blog  http://charith.wickramaarachchi.org/ <http://charithwiki.blogspot.com/>
> Twitter  @charithwiki <https://twitter.com/charithwiki>
> 
> This communication may contain privileged or other confidential information and is intended
exclusively for the addressee/s. If you are not the intended recipient/s, or believe that
you may have
> received this communication in error, please reply to the sender indicating that fact
and delete the copy you received and in addition, you should not print, copy, retransmit,
disseminate, or otherwise use the information contained in this communication. Internet communications
cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept
liability for any errors or omissions


Mime
View raw message