flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2236) RowSerializer and CaseClassComparator are not in sync regarding Null-Values
Date Thu, 05 Nov 2015 09:15:28 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991375#comment-14991375
] 

Aljoscha Krettek commented on FLINK-2236:
-----------------------------------------

The {{RowSerializer}} is already independent of the {{CaseClassSerializer}} so it could be
extended to be able to handle null values. The only problem is that we have to find a way
to get data into {{Row}}s in the first place. Right now a Table is always created from a Tuple
or Pojo {{DataSet}} and Tuples don't support null values.

> RowSerializer and CaseClassComparator are not in sync regarding Null-Values
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-2236
>                 URL: https://issues.apache.org/jira/browse/FLINK-2236
>             Project: Flink
>          Issue Type: Bug
>    Affects Versions: 0.10
>            Reporter: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: 0.10
>
>
> The RowSerializer was recently updated to allow it to handle null values. This changes
the binary layout of the serialised data. CaseClassComparator, which is used for comparison,
is not aware of this new layout and therefore fails. The problem only occurs when a key is
long enough to exceed the normalised-key length, that's why the tests fail to notice the bug.
> I think the solution is to modify all Tuple-like serializers/comparators (TupleComparatorBase,
CaseClassComparator, TupleSerializer, CaseClassSerializer, RowSerializer) to handle null-values,
thus bringing the binary format in sync again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message