flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Adding flink-scala as a dependency to flink-streaming-core
Date Wed, 10 Jun 2015 13:41:20 GMT
I see the reason why you want to add flink-scala as a dependency to
flink-streaming-core. However, it does not feel right to add an API package
to a core package IMHO.

But I noticed that flink-streaming-core also depends on flink-java. Which
seems odd to me as well. I'm not a streaming expert and thus cannot tell
much about the reasons why a core package has a dependency on an API
package but for me this looks more like an indicator for a necessary
restructuring of our packages. Maybe someone working on the streaming parts
can chime in and shed some light on the required dependencies.

Cheers,
Till

On Wed, Jun 10, 2015 at 2:13 PM Gábor Gévay <ggab90@gmail.com> wrote:

> Hello,
>
> I would like to ask if it would be OK if I added flink-scala as a
> dependency to flink-streaming-core. An alternative would be to move
> the Scala typeutils to flink-core (to where the Java typeutils are).
> Why I need this:
>
> While I am implementing the fast median calculation for windows as
> part of my Google Summer of Code project, I am refactoring the way
> sum, min, max, etc. are accessing the user-specified field
> (https://github.com/apache/flink/pull/684). Currently both the logic
> of their aggregators are duplicated for the different kinds of types
> (tuple, pojo, array, Scala case class, simple), and also the field
> access logic is duplicated across the different aggregators. In my
> GSoC project I will implement some further methods (avg, variance,
> etc.) that take the same kind of parameters as sum, min, etc., so it
> will be neccassary to have the field access logic centralized (this is
> the FieldAccessor class in the PR). It would be convenient if this
> could also handle Scala case classes, for which CaseClassTypeInfo is
> needed which is currently in flink-scala.
>
> Best regards,
> Gabor
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message