flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Adding flink-scala as a dependency to flink-streaming-core
Date Wed, 10 Jun 2015 13:55:28 GMT
As for the current streaming dependencies: flink-java is not needed at all,
flink-clients is only needed in specific cases. Thanks for spotting it, I
am cleaning it.

As for Gabor's problem: maybe we should move the CaseClassTypeInfo to
flink-core then given Till's comment. Any opinions on that?

On Wed, Jun 10, 2015 at 3:46 PM, Till Rohrmann <trohrmann@apache.org> wrote:

> Btw: I noticed that all streaming modules depend on flink-core,
> flink-runtime, flink-clients and flink-java. Is there a particular reason
> why the streaming connectors depend on flink-clients and flink-java?
>
> On Wed, Jun 10, 2015 at 3:41 PM Till Rohrmann <trohrmann@apache.org>
> wrote:
>
> > I see the reason why you want to add flink-scala as a dependency to
> > flink-streaming-core. However, it does not feel right to add an API
> package
> > to a core package IMHO.
> >
> > But I noticed that flink-streaming-core also depends on flink-java. Which
> > seems odd to me as well. I'm not a streaming expert and thus cannot tell
> > much about the reasons why a core package has a dependency on an API
> > package but for me this looks more like an indicator for a necessary
> > restructuring of our packages. Maybe someone working on the streaming
> parts
> > can chime in and shed some light on the required dependencies.
> >
> > Cheers,
> > Till
> >
> >
> > On Wed, Jun 10, 2015 at 2:13 PM Gábor Gévay <ggab90@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> I would like to ask if it would be OK if I added flink-scala as a
> >> dependency to flink-streaming-core. An alternative would be to move
> >> the Scala typeutils to flink-core (to where the Java typeutils are).
> >> Why I need this:
> >>
> >> While I am implementing the fast median calculation for windows as
> >> part of my Google Summer of Code project, I am refactoring the way
> >> sum, min, max, etc. are accessing the user-specified field
> >> (https://github.com/apache/flink/pull/684). Currently both the logic
> >> of their aggregators are duplicated for the different kinds of types
> >> (tuple, pojo, array, Scala case class, simple), and also the field
> >> access logic is duplicated across the different aggregators. In my
> >> GSoC project I will implement some further methods (avg, variance,
> >> etc.) that take the same kind of parameters as sum, min, etc., so it
> >> will be neccassary to have the field access logic centralized (this is
> >> the FieldAccessor class in the PR). It would be convenient if this
> >> could also handle Scala case classes, for which CaseClassTypeInfo is
> >> needed which is currently in flink-scala.
> >>
> >> Best regards,
> >> Gabor
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message