flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt <dromitl...@gmail.com>
Subject Re: Serializing NULLs
Date Thu, 22 Dec 2016 13:56:34 GMT
Here is the code of a Double wrapper with null support [1].

[1] https://gist.github.com/a8e8aa377957d3d51eadf36fe5c92a9e

On Tue, Dec 20, 2016 at 4:26 PM, Anirudh Mallem <anirudh.mallem@247-inc.com>
wrote:

> If you are using Avro generated classes then you cannot have your values
> null.
> https://cwiki.apache.org/confluence/display/AVRO/FAQ#FAQ-Whyisn'
> teveryvalueinAvronullable?
>
> From: Stephan Ewen
> Reply-To: "user@flink.apache.org"
> Date: Tuesday, December 20, 2016 at 8:17 AM
> To: "user@flink.apache.org"
> Subject: Re: Serializing NULLs
>
> Thanks for sharing the stack trace.
>
> This seems not really Flink related, it is part of the specific Avro
> encoding logic.
> The Avro Generic Record Type apparently does not allow the map value to be
> null.
>
>
>
> On Tue, Dec 20, 2016 at 4:55 PM, Matt <dromitlabs@gmail.com> wrote:
>
>> Here is the back trace: https://gist.github.com
>> /56af4818bcf5dee6b97c248fd9233c67
>>
>> In the meanwhile I've solved the issue by creating a POJO class where
>> null is just Long.MIN_VALUE, that with a custom equals() made the trick. I
>> guess it's not as fast as de/serializing Double though.
>>
>> If you need any other information let me know.
>>
>> Regards,
>> Matt
>>
>> On Tue, Dec 20, 2016 at 6:46 AM, Stephan Ewen <sewen@apache.org> wrote:
>>
>>> The "null" support in some types is not fully developed. However in that
>>> case I am wondering why it does not work. Can you share the stack trace, so
>>> we can take a look at the serializer?
>>>
>>>
>>>
>>> On Mon, Dec 19, 2016 at 9:56 PM, Matt <dromitlabs@gmail.com> wrote:
>>>
>>>> Hello list,
>>>>
>>>> I'm getting this error:
>>>>
>>>>
>>>> *java.lang.RuntimeException: Could not forward element to next operator
>>>> *
>>>> *...*
>>>> *Caused by: java.lang.NullPointerException: in com.entities.Sector in
>>>> map in double null of double of map in field properties of
>>>> com.entities.Sector*
>>>> *...*
>>>> *Caused by: java.lang.NullPointerException*
>>>>
>>>> The field mentioned is a HashMap<String, Double>, and some keys are
>>>> mapped to null values.
>>>>
>>>> Why isn't it possible to forward/serialize those elements with null
>>>> values?
>>>> What do you do when your elements may contain nulls?
>>>>
>>>> Regards,
>>>> Matt
>>>>
>>>
>>>
>>
>

Mime
View raw message