Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAJZ2dcW5JK3khtndSu4ixPfe5WgHOj-=iCJaUN3pdhK1DY_6ZQ@mail.gmail.com>
References: 
 <CAJZ2dcW5JK3khtndSu4ixPfe5WgHOj-=iCJaUN3pdhK1DY_6ZQ@mail.gmail.com>
From: Fabian Hueske <fhueske@gmail.com>
Date: Wed, 16 Sep 2015 16:50:38 +0200
Message-ID: 
 <CAAdrtT3upgK-JXbaJ8MZ1_ZFbbWjM+VVPo0S0nCAZKwtrisUEg@mail.gmail.com>
Subject: Re: CompactingHashTable question
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=001a113494dac484ed051fde6d39

--001a113494dac484ed051fde6d39
Content-Type: text/plain; charset=UTF-8

Hi Vasia,

I looked into the code. A serializer should never return null when
deserializing. Either it does not detect that something went wrong with the
deserialization or it should throw an exception.

Regarding the handling of null returns in the Drivers. If there is no entry
in the HT for a certain key, the HT will return null which is expected.
If a CoGroupWithSolutionSet*Driver receives a null value, it gives an empty
iterator to the user function. The JoinWithSolutionSet*Driver calls the
join function with a null value. Both behaviors are expected. A join with a
solution set is actually an outer join and a join function in such a join
needs to be able to handle null values on the solution set side.

Cheers, Fabian


2015-09-15 17:41 GMT+02:00 Vasiliki Kalavri <vasilikikalavri@gmail.com>:

> Hello to my squirrels,
>
> I ran into an NPE for some iterations code and it looks like what's
> described in FLINK-2443 <https://issues.apache.org/jira/browse/FLINK-2443
> >.
> I'm trying to understand the problem and I could really use your help :)
>
> So far, it seems that the exception is caused by a null value returned by
> CompactingHashTable.*getMatchFor*(PT probeSideRecord).
>
> This method returns null in the following cases:
> - when the hash table is "closed"
> - when the segment is done
> - if the serializer actually returns a null record
>
> It seems that on the join/cogroup driver side there is no check or special
> handling when the build side record is null, i.e. the null record is still
> passed to the join function.
> Is this correct and if not, what should the driver do in this case?
>
> Thank you!
>
> Cheers,
> Vasia.
>

--001a113494dac484ed051fde6d39--