flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [DISCUSS] Merging Storm compatibility to Flink-contrib
Date Wed, 13 May 2015 07:46:31 GMT
That would be great, thank you!

Please write down everything you see and post it on the mailing list.

On Wed, May 13, 2015 at 9:42 AM, Szabó Péter <nemderogatorius@gmail.com>
wrote:

> @Robert: I have a little storm experience. I will try to run some examples
> on the cluster.
>
> Peter
>
> 2015-05-12 11:40 GMT+02:00 Matthias J. Sax <mjsax@informatik.hu-berlin.de
> >:
>
> > Hi,
> >
> > some UnsupportedOperationExceptions are required, because Storm
> > interfaces are implement but Flink cannot support those functionality.
> > Some other are "not yet implemented" once.
> >
> > A few other of them could be removed (in case an interface in not
> > implemented, but only mimicked), by removing the whole method. I prefer
> > to mimic interfaced completely and through
> > UnsupportedOperationExceptions, because if a user wants to execute Storm
> > code on Flink, less changes to the original Storm code are necessary,
> > making the transition easier.
> >
> > Of course, it is a valid argument to remove the methods completely to
> > raise incompatibilities directly are compile time.
> >
> > The TODOs are points that might or might not be Flink compatible. This
> > must be checked and maybe discussed.
> >
> > Any feedback is welcome.
> >
> >
> > -Matthias
> >
> > On 05/12/2015 10:46 AM, Robert Metzger wrote:
> > > Hi,
> > >
> > > Thank you for starting the discussion Marton!
> > >
> > > I would really like to merge the storm compat to our source repo. I
> think
> > > that code which is not merged there will not get enough attention.
> > >
> > > I'm against splitting flink-contrib into small maven modules. I totally
> > > understand your reasoning (mixed dependencies), but "flink-staging"
> > exists
> > > exactly for that purpose (one maven module per "beta"-module).
> > > For now, users depending on flink-contrib have to define exclusions to
> > > control the dependencies.
> > >
> > > So I'm +1 for merging it to "flink-contrib".
> > > I guess not all committers have time to look into the pull request,
> > > therefore, I want to remind you that the code contains a lot of TODOs
> and
> > > UnsupportedOperationExceptions.
> > >
> > > @Marton: Do you know anybody from the Budapest Flink Streaming crew
> with
> > > some Storm experience who could try out the code on a cluster and give
> > some
> > > feedback?
> > >
> > >
> > > On Tue, May 12, 2015 at 9:52 AM, Márton Balassi <
> > balassi.marton@gmail.com>
> > > wrote:
> > >
> > >> The purpose of flink-contrib currently is to hold contributions to the
> > >> project that we do not consider part of the core flink functionality,
> > but
> > >> provide useful tools around it. In general code placed here has to
> meet
> > >> less requirements in terms of covering all corner cases if it
> provides a
> > >> nice solution for a set of well defined problems.
> > >>
> > >> As of today it has two small utilities, the TweetInputFormat (by
> Mustafa
> > >> Elbehery) and the collect functionality for the DataStream (by Gabor
> > >> Gevay).
> > >>
> > >> The pull request for the Storm compatibility layer (by Matthias J.
> Sax)
> > [1]
> > >> raises the issue as it is way more code to maintain and is more
> complex
> > in
> > >> general that how the community would like to handle these in terms of
> > >> distribution. Do we want to have it in the Flink repository or maybe
> in
> > a
> > >> separate one.
> > >>
> > >> I am personally really for having the Storm compatibility layer under
> > >> flink-contrib as Matthias is very active on the mailing list and has
> > also
> > >> expressed his interest of further developing the functionality of the
> > >> compatibility layer. To top that a couple of users got excited about
> the
> > >> new feature, so I see no risk in having this code in the main
> > repository.
> > >>
> > >> As for the structure of flink-contrib I would have the contents
> > separated
> > >> to slim as possible maven projects, to make sure that the users only
> get
> > >> the dependencies that they really need.
> > >>
> > >> [1] https://github.com/apache/flink/pull/573
> > >>
> > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message