flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Adding flink-scala as a dependency to flink-streaming-core
Date Wed, 10 Jun 2015 13:41:20 GMT
I see the reason why you want to add flink-scala as a dependency to
flink-streaming-core. However, it does not feel right to add an API package
to a core package IMHO.

But I noticed that flink-streaming-core also depends on flink-java. Which
seems odd to me as well. I'm not a streaming expert and thus cannot tell
much about the reasons why a core package has a dependency on an API
package but for me this looks more like an indicator for a necessary
restructuring of our packages. Maybe someone working on the streaming parts
can chime in and shed some light on the required dependencies.


On Wed, Jun 10, 2015 at 2:13 PM Gábor Gévay <ggab90@gmail.com> wrote:

> Hello,
> I would like to ask if it would be OK if I added flink-scala as a
> dependency to flink-streaming-core. An alternative would be to move
> the Scala typeutils to flink-core (to where the Java typeutils are).
> Why I need this:
> While I am implementing the fast median calculation for windows as
> part of my Google Summer of Code project, I am refactoring the way
> sum, min, max, etc. are accessing the user-specified field
> (https://github.com/apache/flink/pull/684). Currently both the logic
> of their aggregators are duplicated for the different kinds of types
> (tuple, pojo, array, Scala case class, simple), and also the field
> access logic is duplicated across the different aggregators. In my
> GSoC project I will implement some further methods (avg, variance,
> etc.) that take the same kind of parameters as sum, min, etc., so it
> will be neccassary to have the field access logic centralized (this is
> the FieldAccessor class in the PR). It would be convenient if this
> could also handle Scala case classes, for which CaseClassTypeInfo is
> needed which is currently in flink-scala.
> Best regards,
> Gabor

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message