flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Adding non-core API features to Flink
Date Sun, 25 Jan 2015 19:55:13 GMT
+1 to what robert said

On Sun, Jan 25, 2015 at 8:54 PM, Kostas Tzoumas <ktzoumas@apache.org> wrote:
> +1 for a package repository in the long run as per Ted's suggestion, and
> moving forward with what Robert proposes
>
> On Sun, Jan 25, 2015 at 10:33 AM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> I'm also in favor of option 1) with a "flink-contrib" maven module.
>>
>> I agree with Ted that we should certainly think about establishing a highly
>> visible, easy to contribute and easy to use infrastructure for all kinds of
>> contributions around the project.
>> But I suspect that we need some time to come up with a good architecture
>> and infrastructure for that. Maybe this also comes as an outside
>> contribution to Flink?
>>
>> To have something immediately, we should start with a "flink-contrib"
>> module.
>>
>> One thing that I would like to discuss first is a clear set of rules for
>> contributions into that module.
>> Code contributions to "flink-contrib" need:
>> - to be tested on a cluster (not only by single-jvm tests)
>> - to have test cases (because otherwise we can not guarantee that they
>> build with our changes
>> - to be of use for others
>> - to have some documentation
>>
>> I would not deploy the flink-contrib package in the standard flink
>> distribution. Users will have to add them as a maven dependency.
>>
>>
>>
>>
>> On Sat, Jan 24, 2015 at 11:02 PM, Ted Dunning <ted.dunning@gmail.com>
>> wrote:
>>
>> > As the community of flink add-ons grows, a CPAN or maven-like mechanism
>> > might be a nice option.  That would let people download and install
>> > extensions very fluidly.
>> >
>> > The argument for making Apache contributions is definitely valid, but the
>> > argument for the agility of fostering independent projects is that
>> projects
>> > can gain lots of popularity very quickly this way.  CPAN, CRAN, pip,
>> maven
>> > and RubyGems can be argued to be critical components of the popularity of
>> > Perl, R, Python, Java/Scala and Ruby respectively.
>> >
>> >
>> >
>> >
>> > On Sat, Jan 24, 2015 at 4:26 PM, Fabian Hueske <fhueske@gmail.com>
>> wrote:
>> >
>> > > I am also more in favor of option 1).
>> > >
>> > > 2015-01-24 20:27 GMT+01:00 Kostas Tzoumas <ktzoumas@apache.org>:
>> > >
>> > > > Thanks Fabian for starting the discussion.
>> > > >
>> > > > I would be biased towards option (1) that Stephan highlighted for
the
>> > > > following reasons:
>> > > >
>> > > > - A separate github project is one more infrastructure to manage,
and
>> > it
>> > > > lives outside the ASF. I would like to bring as much code as possible
>> > to
>> > > > the Apache Software Foundation, and not divide the codebase into two
>> > > > separate repositories.
>> > > >
>> > > > - The personal gratification (and thus motivation) is higher when
>> > > > contributing to a top-level Apache project than a github repository
>> > > > slightly associated with an ASF project. And contributors to the
>> Flink
>> > > > project get karma that may lead to new committers, which is crucial
>> as
>> > > the
>> > > > project is growing.
>> > > >
>> > > > Of course, non Apache-licensed contributions cannot be accepted. If
>> we
>> > > have
>> > > > a good amount of those, we can start an infrastructure for Flink
>> > packages
>> > > > that lives outside the ASF, but I would wait for the need to come
>> > before
>> > > > doing this.
>> > > >
>> > > > My proposal would be to funnel contributions to the main repository
>> > (in a
>> > > > flink-contrib module) for now, including the recent contributions.
>> > > >
>> > > > Kostas
>> > > >
>> > > >
>> > > > On Sat, Jan 24, 2015 at 11:15 AM, Stephan Ewen <sewen@apache.org>
>> > wrote:
>> > > >
>> > > > > Yes, a "flink-contrib" project would be great.
>> > > > >
>> > > > > We have two options:
>> > > > >
>> > > > > 1) Make it part of the flink apache project.
>> > > > >   - PRO this makes it easy to get stuff for users
>> > > > >   - CONTRA this means stronger requirements on the code, blocker
>> for
>> > > code
>> > > > > that uses dependencies under certain licenses, etc.
>> > > > >
>> > > > > 2) Make an independent github project.
>> > > > >  - PRO contributions can depended on more licenses, such as LGPL
>> > > > >  - PRO we can have more people that commit to this repo, committers
>> > can
>> > > > be
>> > > > > different from flink committers
>> > > > >  - CONTRA people need to grab the extensions from a different
>> > location
>> > > > >
>> > > > >
>> > > > > I am slightly biased towards (2), but open to both.
>> > > > >
>> > > > > Stephan
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Sat, Jan 24, 2015 at 5:29 AM, Chiwan Park <
>> chiwanpark@icloud.com>
>> > > > > wrote:
>> > > > >
>> > > > > > I think top level maven module called "flink-contrib" is
>> > reasonable.
>> > > > > There
>> > > > > > are other projects having contrib package such as Akka,
Django.
>> > > > > >
>> > > > > > Regards, Chiwan Park (Sent with iPhone)
>> > > > > >
>> > > > > > 2015. 1. 24. 오후 7:15 Fabian Hueske <fhueske@gmail.com>
작성:
>> > > > > >
>> > > > > > > Hi all,
>> > > > > > >
>> > > > > > > we got a few contribution requests lately to add cool
but
>> > > "non-core"
>> > > > > > > features to our API.
>> > > > > > > In previous discussions, concerns were raised to not
bloat the
>> > APIs
>> > > > > with
>> > > > > > > too many "shortcut", "syntactic sugar", or special-case
>> features.
>> > > > > > >
>> > > > > > > Instead we could setup a place to add Input/OutputFormats,
>> common
>> > > > > > > operations, etc. which does not need as much control
as the
>> core
>> > > > APIs.
>> > > > > > Open
>> > > > > > > questions are:
>> > > > > > > - How do we organize it? (top-level maven module, modules
in
>> > > > > flink-java,
>> > > > > > > flink-scala, java packages in the API modules, ...)
>> > > > > > > - How do we name it? flink-utils, flink-packages, ...
>> > > > > > >
>> > > > > > > Any opinions on this?
>> > > > > > >
>> > > > > > > Cheers, Fabian
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>

Mime
View raw message