flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Adding non-core API features to Flink
Date Mon, 26 Jan 2015 11:00:40 GMT
I've added a JIRA issue to create the module:
https://issues.apache.org/jira/browse/FLINK-1452


On Mon, Jan 26, 2015 at 11:39 AM, Till Rohrmann <trohrmann@apache.org>
wrote:

> +1 for Robert's proposal.
>
> On Mon, Jan 26, 2015 at 5:43 AM, Henry Saputra <henry.saputra@gmail.com>
> wrote:
>
> > +1 for the idea.
> >
> > We need to make sure PMC of Flink maintains knowledge of standard
> > Flink distribution, hence the "flink-contrib" should not be part of
> > the release.
> >
> > - Henry
> >
> > On Sun, Jan 25, 2015 at 10:33 AM, Robert Metzger <rmetzger@apache.org>
> > wrote:
> > > I'm also in favor of option 1) with a "flink-contrib" maven module.
> > >
> > > I agree with Ted that we should certainly think about establishing a
> > highly
> > > visible, easy to contribute and easy to use infrastructure for all
> kinds
> > of
> > > contributions around the project.
> > > But I suspect that we need some time to come up with a good
> architecture
> > > and infrastructure for that. Maybe this also comes as an outside
> > > contribution to Flink?
> > >
> > > To have something immediately, we should start with a "flink-contrib"
> > > module.
> > >
> > > One thing that I would like to discuss first is a clear set of rules
> for
> > > contributions into that module.
> > > Code contributions to "flink-contrib" need:
> > > - to be tested on a cluster (not only by single-jvm tests)
> > > - to have test cases (because otherwise we can not guarantee that they
> > > build with our changes
> > > - to be of use for others
> > > - to have some documentation
> > >
> > > I would not deploy the flink-contrib package in the standard flink
> > > distribution. Users will have to add them as a maven dependency.
> > >
> > >
> > >
> > >
> > > On Sat, Jan 24, 2015 at 11:02 PM, Ted Dunning <ted.dunning@gmail.com>
> > wrote:
> > >
> > >> As the community of flink add-ons grows, a CPAN or maven-like
> mechanism
> > >> might be a nice option.  That would let people download and install
> > >> extensions very fluidly.
> > >>
> > >> The argument for making Apache contributions is definitely valid, but
> > the
> > >> argument for the agility of fostering independent projects is that
> > projects
> > >> can gain lots of popularity very quickly this way.  CPAN, CRAN, pip,
> > maven
> > >> and RubyGems can be argued to be critical components of the popularity
> > of
> > >> Perl, R, Python, Java/Scala and Ruby respectively.
> > >>
> > >>
> > >>
> > >>
> > >> On Sat, Jan 24, 2015 at 4:26 PM, Fabian Hueske <fhueske@gmail.com>
> > wrote:
> > >>
> > >> > I am also more in favor of option 1).
> > >> >
> > >> > 2015-01-24 20:27 GMT+01:00 Kostas Tzoumas <ktzoumas@apache.org>:
> > >> >
> > >> > > Thanks Fabian for starting the discussion.
> > >> > >
> > >> > > I would be biased towards option (1) that Stephan highlighted
for
> > the
> > >> > > following reasons:
> > >> > >
> > >> > > - A separate github project is one more infrastructure to manage,
> > and
> > >> it
> > >> > > lives outside the ASF. I would like to bring as much code as
> > possible
> > >> to
> > >> > > the Apache Software Foundation, and not divide the codebase into
> two
> > >> > > separate repositories.
> > >> > >
> > >> > > - The personal gratification (and thus motivation) is higher
when
> > >> > > contributing to a top-level Apache project than a github
> repository
> > >> > > slightly associated with an ASF project. And contributors to
the
> > Flink
> > >> > > project get karma that may lead to new committers, which is
> crucial
> > as
> > >> > the
> > >> > > project is growing.
> > >> > >
> > >> > > Of course, non Apache-licensed contributions cannot be accepted.
> If
> > we
> > >> > have
> > >> > > a good amount of those, we can start an infrastructure for Flink
> > >> packages
> > >> > > that lives outside the ASF, but I would wait for the need to
come
> > >> before
> > >> > > doing this.
> > >> > >
> > >> > > My proposal would be to funnel contributions to the main
> repository
> > >> (in a
> > >> > > flink-contrib module) for now, including the recent contributions.
> > >> > >
> > >> > > Kostas
> > >> > >
> > >> > >
> > >> > > On Sat, Jan 24, 2015 at 11:15 AM, Stephan Ewen <sewen@apache.org>
> > >> wrote:
> > >> > >
> > >> > > > Yes, a "flink-contrib" project would be great.
> > >> > > >
> > >> > > > We have two options:
> > >> > > >
> > >> > > > 1) Make it part of the flink apache project.
> > >> > > >   - PRO this makes it easy to get stuff for users
> > >> > > >   - CONTRA this means stronger requirements on the code,
blocker
> > for
> > >> > code
> > >> > > > that uses dependencies under certain licenses, etc.
> > >> > > >
> > >> > > > 2) Make an independent github project.
> > >> > > >  - PRO contributions can depended on more licenses, such
as LGPL
> > >> > > >  - PRO we can have more people that commit to this repo,
> > committers
> > >> can
> > >> > > be
> > >> > > > different from flink committers
> > >> > > >  - CONTRA people need to grab the extensions from a different
> > >> location
> > >> > > >
> > >> > > >
> > >> > > > I am slightly biased towards (2), but open to both.
> > >> > > >
> > >> > > > Stephan
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Sat, Jan 24, 2015 at 5:29 AM, Chiwan Park <
> > chiwanpark@icloud.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I think top level maven module called "flink-contrib"
is
> > >> reasonable.
> > >> > > > There
> > >> > > > > are other projects having contrib package such as Akka,
> Django.
> > >> > > > >
> > >> > > > > Regards, Chiwan Park (Sent with iPhone)
> > >> > > > >
> > >> > > > > 2015. 1. 24. 오후 7:15 Fabian Hueske <fhueske@gmail.com>
작성:
> > >> > > > >
> > >> > > > > > Hi all,
> > >> > > > > >
> > >> > > > > > we got a few contribution requests lately to add
cool but
> > >> > "non-core"
> > >> > > > > > features to our API.
> > >> > > > > > In previous discussions, concerns were raised
to not bloat
> the
> > >> APIs
> > >> > > > with
> > >> > > > > > too many "shortcut", "syntactic sugar", or special-case
> > features.
> > >> > > > > >
> > >> > > > > > Instead we could setup a place to add Input/OutputFormats,
> > common
> > >> > > > > > operations, etc. which does not need as much control
as the
> > core
> > >> > > APIs.
> > >> > > > > Open
> > >> > > > > > questions are:
> > >> > > > > > - How do we organize it? (top-level maven module,
modules in
> > >> > > > flink-java,
> > >> > > > > > flink-scala, java packages in the API modules,
...)
> > >> > > > > > - How do we name it? flink-utils, flink-packages,
...
> > >> > > > > >
> > >> > > > > > Any opinions on this?
> > >> > > > > >
> > >> > > > > > Cheers, Fabian
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message