Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 60E21200BE7 for ; Tue, 20 Dec 2016 17:17:51 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 5F83D160B29; Tue, 20 Dec 2016 16:17:51 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A85C6160B12 for ; Tue, 20 Dec 2016 17:17:50 +0100 (CET) Received: (qmail 98760 invoked by uid 500); 20 Dec 2016 16:17:44 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 98751 invoked by uid 99); 20 Dec 2016 16:17:44 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Dec 2016 16:17:44 +0000 Received: from mail-io0-f179.google.com (mail-io0-f179.google.com [209.85.223.179]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 8CDA51A0046 for ; Tue, 20 Dec 2016 16:17:44 +0000 (UTC) Received: by mail-io0-f179.google.com with SMTP id d9so183118347ioe.0 for ; Tue, 20 Dec 2016 08:17:44 -0800 (PST) X-Gm-Message-State: AIkVDXJZMFAsSh7cecCzGkAtu1dUdwDnRVlg9g/vNnx9Vl60kKfKuWrzGef/kejLCv/GWJCHUi6n2VhjZ1CvfA== X-Received: by 10.107.170.231 with SMTP id g100mr873919ioj.32.1482250663885; Tue, 20 Dec 2016 08:17:43 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.48.18 with HTTP; Tue, 20 Dec 2016 08:17:43 -0800 (PST) In-Reply-To: References: From: Stephan Ewen Date: Tue, 20 Dec 2016 17:17:43 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Serializing NULLs To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a1142e6b24a12380544196078 archived-at: Tue, 20 Dec 2016 16:17:51 -0000 --001a1142e6b24a12380544196078 Content-Type: text/plain; charset=UTF-8 Thanks for sharing the stack trace. This seems not really Flink related, it is part of the specific Avro encoding logic. The Avro Generic Record Type apparently does not allow the map value to be null. On Tue, Dec 20, 2016 at 4:55 PM, Matt wrote: > Here is the back trace: https://gist.github.com/ > 56af4818bcf5dee6b97c248fd9233c67 > > In the meanwhile I've solved the issue by creating a POJO class where null > is just Long.MIN_VALUE, that with a custom equals() made the trick. I guess > it's not as fast as de/serializing Double though. > > If you need any other information let me know. > > Regards, > Matt > > On Tue, Dec 20, 2016 at 6:46 AM, Stephan Ewen wrote: > >> The "null" support in some types is not fully developed. However in that >> case I am wondering why it does not work. Can you share the stack trace, so >> we can take a look at the serializer? >> >> >> >> On Mon, Dec 19, 2016 at 9:56 PM, Matt wrote: >> >>> Hello list, >>> >>> I'm getting this error: >>> >>> >>> *java.lang.RuntimeException: Could not forward element to next operator* >>> *...* >>> *Caused by: java.lang.NullPointerException: in com.entities.Sector in >>> map in double null of double of map in field properties of >>> com.entities.Sector* >>> *...* >>> *Caused by: java.lang.NullPointerException* >>> >>> The field mentioned is a HashMap, and some keys are >>> mapped to null values. >>> >>> Why isn't it possible to forward/serialize those elements with null >>> values? >>> What do you do when your elements may contain nulls? >>> >>> Regards, >>> Matt >>> >> >> > --001a1142e6b24a12380544196078 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for sharing the stack trace.

This seems not really Flink related, it is part of the specific Avr= o encoding logic.
The Avro Generic Reco= rd Type apparently does not allow the map value to be null.



On Tue, Dec 20, 2016 at 4:55 PM, Matt <dromitlabs@gmail.com> wrote:
In the meanwhile I've solved the issue by creating a POJO c= lass where null is just Long.MIN_VALUE, that with a custom equals() made th= e trick. I guess it's not as fast as de/serializing Double though.

If you need any other information let me know.

Regards,
Matt

On Tue, Dec 20, 2016 at 6:46 AM, Stephan Ewen <sewen@apach= e.org> wrote:
The "null" support in some types is n= ot fully developed. However in that case I am wondering why it does not wor= k. Can you share the stack trace, so we can take a look at the serializer?<= br>

=


On Mon, Dec 19, 2016 at 9:56 PM, Matt <dromitlabs@gmail.com> wrote:
Hello list,

I'm getting = this error:

java.lang.RuntimeException: Could n= ot forward element to next operator
...
=
Caused by: java.lang.NullPointerException: in com.entities.Sec= tor in map in double null of double of map in field properties of com.entit= ies.Sector
...
Caused by: java= .lang.NullPointerException

The field men= tioned is a HashMap<String, Double>, and some keys are mapped to null= values.

Why isn't it possible to forward/seri= alize those elements with null values?
What do you do when your e= lements may contain nulls?

Regards,
Matt=



--001a1142e6b24a12380544196078--