flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Connolly <stephen.alan.conno...@gmail.com>
Subject Re: Why don't Tuple types implement Comparable?
Date Fri, 22 Feb 2019 10:16:00 GMT
On Thu, 21 Feb 2019 at 18:29, Frank Grimes <frankgrimes97@yahoo.com> wrote:

> Hi,
>
> I've recently started to evaluate Flink and have found it odd that its
> Tuple types, while Serializable, don't implement java.lang.Comparable.
> This means that I either need to provide an KeySelector for many
> operations or subtype the Tuple types and provide my own implementation of
> compareTo for each.
>
> Is there a technical reason why this was omitted?
>

A Tuple1<T0> would need to have an implementation of the comparable
contract in order to be comparable. The only way I can see of writing that
would be if we had:

Tuple1<T0 extends Comparable<? super T0>>

Similarly you'd have

Tuple25<T0 extends Comparable<? super T0>, T1 extends Comparable<? super
T1>, T2 extends Comparable<? super T2>, ..., T24 extends Comparable<? super
T24>>

Which is a lot less screen friendly than say

Tuple25<T0, T1, T2, ..., T24>

So from a purely code readability PoV you would need to at least replace
the  Comparable<? super T> with Comparable<T> with the immediate reduction
of utility... or perhaps you would remove the generics entirely and go for
just Comparable and have the implementation just follow the guarantee...
except javac will error out for the erasure.

Now jOOQ just ignores the comparability contract on the generic types, e.g.
https://github.com/jOOQ/jOOL/blob/master/jOOL/src/main/java/org/jooq/lambda/tuple/Tuple1.java#L35

The result is that when asked to compare you get a cast that will work or
fail:
https://github.com/jOOQ/jOOL/blob/master/jOOL/src/main/java/org/jooq/lambda/tuple/Tuples.java#L31

So jOOQ is following the "fail at runtime" mode... you can make tuples of
non-comparable objects and only at runtime if you try to compare them will
it blow up.

Flink is following the "fail at compile time" mode... if you try to compare
a tuple, you need to provide a way to compare them.

Flink does a lot of work - from what I can see - to fail early and fast.
For example, if type inference fails on the dataflow plan, it will error
out fast. I would hate to have a Tuple3<A,B,Y> where A and B are both
Comparable and Y is not... my tests happened to have a small subset of A
and B values that never resulted in a comparison of Y values... so the
tests didn't fail... but now on the production environment... at runtime...
1 month later in the middle of my vacation I get a call because the
topology is stuck and failing on (a,b,y1).compareTo((a,b,y2))

So my analysis would say it is not a technical reason but rather a
principal reason of "fail fast"


>
> For example, the JOOQ/JOOL Tuple types all implement Comparable:
>
> https://github.com/jOOQ/jOOL/blob/master/jOOL-java-8/src/main/java/org/jooq/lambda/tuple/Tuple2.java#L39
>
> As an aside, I tried replacing usage of Flink's Tuple types with the JOOL
> ones but they caused a StackOverflowError similar to this one:
> https://issues.apache.org/jira/browse/FLINK-3922
>
> Thanks,
>
> Frank Grimes
>
>

Mime
View raw message