flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charith Wickramarachchi <charith.dhanus...@gmail.com>
Subject Re: HashMap/HashSet Serialization Issue
Date Sat, 07 Jan 2017 20:09:56 GMT
Thanks very much.

Regards,
Charith

On Sat, Jan 7, 2017 at 3:42 AM, Jark Wu <wuchong.wc@alibaba-inc.com> wrote:

> Hi Charith,
>
> The LOG message has nothing to worry about. The HashMap and HashSet
> members in your POJO is not a PojoType,
> so they will be handled as GenericType, i.e. will be
> serialized/deserialized by Kryo. But your custom POJO type will still
> recognized as PojoType, i.e. will be serialized/deserialized by Flink
> serialization framework.
>
> In a word, your custom POJO can be serialized/deserialized correctly.
> There is no issue in it.  :)
>
> - Jark Wu
>
> 在 2017年1月7日,下午12:02,Charith Wickramarachchi <charith.dhanushka@gmail.com>
> 写道:
>
> Thanks very much.
>
> Regards,
> Charith
>
> On Fri, Jan 6, 2017 at 7:44 PM, Chen Qin <qinnchen@gmail.com> wrote:
>
>> My understanding is HashMap doesn't work with Flink Native serialization
>> framework, though I might be wrong.
>>
>> This might worth reading
>> ​https://cwiki.apache.org/confluence/display/FLINK/Type+Syst
>> em,+Type+Extraction,+Serialization
>>
>> -Chen​
>>
>> On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi <
>> charith.dhanushka@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I am using flink-gelly and using a custom POJO type as the
>>> VertexValue/MessageType (I am using the vertex-centric model). The POJO
>>> contains HashMap/HashSet as members. While executing the job I am getting
>>> following Log message.
>>>
>>> 17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor
>>>           - No fields detected for class java.util.HashSet. Cannot be used
>>> as a PojoType. Will be handled as GenericType
>>> 17:50:53,583 INFO  org.apache.flink.api.java.typeutils.
>>> ​​
>>> TypeExtractor             - class java.util.HashMap is not a valid POJO
>>> type
>>>
>>> Is there a way to resolve this issue?
>>>
>>> Thanks,
>>> Charith
>>>
>>>
>>> --
>>> Charith Dhanushka Wickramaarachchi
>>>
>>> Tel  +1 213 447 4253
>>> Blog  http://charith.wickramaarachchi.org/
>>> <http://charithwiki.blogspot.com/>
>>> Twitter  @charithwiki <https://twitter.com/charithwiki>
>>>
>>> This communication may contain privileged or other confidential information
>>> and is intended exclusively for the addressee/s. If you are not the
>>> intended recipient/s, or believe that you may have
>>> received this communication in error, please reply to the sender indicating
>>> that fact and delete the copy you received and in addition, you should
>>> not print, copy, retransmit, disseminate, or otherwise use the
>>> information contained in this communication. Internet communications
>>> cannot be guaranteed to be timely, secure, error or virus-free. The
>>> sender does not accept liability for any errors or omissions
>>>
>>
>>
>
>
> --
> Charith Dhanushka Wickramaarachchi
>
> Tel  +1 213 447 4253
> Blog  http://charith.wickramaarachchi.org/
> <http://charithwiki.blogspot.com/>
> Twitter  @charithwiki <https://twitter.com/charithwiki>
>
> This communication may contain privileged or other confidential information
> and is intended exclusively for the addressee/s. If you are not the
> intended recipient/s, or believe that you may have
> received this communication in error, please reply to the sender indicating
> that fact and delete the copy you received and in addition, you should
> not print, copy, retransmit, disseminate, or otherwise use the
> information contained in this communication. Internet communications
> cannot be guaranteed to be timely, secure, error or virus-free. The
> sender does not accept liability for any errors or omissions
>
>
>


-- 
Charith Dhanushka Wickramaarachchi

Tel  +1 213 447 4253
Blog  http://charith.wickramaarachchi.org/
<http://charithwiki.blogspot.com/>
Twitter  @charithwiki <https://twitter.com/charithwiki>

This communication may contain privileged or other confidential information
and is intended exclusively for the addressee/s. If you are not the
intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating
that fact and delete the copy you received and in addition, you should not
print, copy, retransmit, disseminate, or otherwise use the information
contained in this communication. Internet communications cannot be
guaranteed to be timely, secure, error or virus-free. The sender does not
accept liability for any errors or omissions

Mime
View raw message