flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [DISCUSS] Merging Storm compatibility to Flink-contrib
Date Tue, 12 May 2015 08:46:36 GMT
Hi,

Thank you for starting the discussion Marton!

I would really like to merge the storm compat to our source repo. I think
that code which is not merged there will not get enough attention.

I'm against splitting flink-contrib into small maven modules. I totally
understand your reasoning (mixed dependencies), but "flink-staging" exists
exactly for that purpose (one maven module per "beta"-module).
For now, users depending on flink-contrib have to define exclusions to
control the dependencies.

So I'm +1 for merging it to "flink-contrib".
I guess not all committers have time to look into the pull request,
therefore, I want to remind you that the code contains a lot of TODOs and
UnsupportedOperationExceptions.

@Marton: Do you know anybody from the Budapest Flink Streaming crew with
some Storm experience who could try out the code on a cluster and give some
feedback?


On Tue, May 12, 2015 at 9:52 AM, Márton Balassi <balassi.marton@gmail.com>
wrote:

> The purpose of flink-contrib currently is to hold contributions to the
> project that we do not consider part of the core flink functionality, but
> provide useful tools around it. In general code placed here has to meet
> less requirements in terms of covering all corner cases if it provides a
> nice solution for a set of well defined problems.
>
> As of today it has two small utilities, the TweetInputFormat (by Mustafa
> Elbehery) and the collect functionality for the DataStream (by Gabor
> Gevay).
>
> The pull request for the Storm compatibility layer (by Matthias J. Sax) [1]
> raises the issue as it is way more code to maintain and is more complex in
> general that how the community would like to handle these in terms of
> distribution. Do we want to have it in the Flink repository or maybe in a
> separate one.
>
> I am personally really for having the Storm compatibility layer under
> flink-contrib as Matthias is very active on the mailing list and has also
> expressed his interest of further developing the functionality of the
> compatibility layer. To top that a couple of users got excited about the
> new feature, so I see no risk in having this code in the main repository.
>
> As for the structure of flink-contrib I would have the contents separated
> to slim as possible maven projects, to make sure that the users only get
> the dependencies that they really need.
>
> [1] https://github.com/apache/flink/pull/573
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message