flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: [DISCUSS] Dropping flink-storm?
Date Thu, 10 Jan 2019 09:15:15 GMT
+1 to drop.

I totally agree with your reasoning. I like that we tried to keep it,
but I don't think the maintenance overhead would be justified.

– Ufuk

On Wed, Jan 9, 2019 at 4:09 PM Till Rohrmann <trohrmann@apache.org> wrote:
>
> With https://issues.apache.org/jira/browse/FLINK-10571, we will remove the
> Storm topologies from Flink and keep the wrappers for the moment.
>
> However, looking at the FlinkTopologyContext [1], it becomes quite obvious
> that Flink's compatibility with Storm is really limited. Almost all of the
> context methods are not supported which makes me wonder how useful these
> wrappers really are. Given the additional maintenance overhead of having
> them in the code base and no indication that someone is actively using
> them, I would still be in favour of removing them. This will reduce our
> maintenance burden in the future. What do you think?
>
> [1]
> https://github.com/apache/flink/blob/master/flink-contrib/flink-storm/src/main/java/org/apache/flink/storm/wrappers/FlinkTopologyContext.java
>
> Cheers,
> Till
>
> On Tue, Oct 9, 2018 at 10:08 AM Fabian Hueske <fhueske@gmail.com> wrote:
>
> > Yes, let's do it this way.
> > The wrapper classes are probably not too complex and can be easily tested.
> > We have the same for the Hadoop interfaces, although I think only the
> > Input- and OutputFormatWrappers are actually used.
> >
> >
> > Am Di., 9. Okt. 2018 um 09:46 Uhr schrieb Chesnay Schepler <
> > chesnay@apache.org>:
> >
> >> That sounds very good to me.
> >>
> >> On 08.10.2018 11:36, Till Rohrmann wrote:
> >> > Good point. The initial idea of this thread was to remove the storm
> >> > compatibility layer completely.
> >> >
> >> > During the discussion I realized that it might be useful for our users
> >> > to not completely remove it in one go. Instead for those who still
> >> > want to use some Bolt and Spout code in Flink, it could be nice to
> >> > keep the wrappers. At least, we could remove flink-storm in a more
> >> > graceful way by first removing the Topology and client parts and then
> >> > the wrappers. What do you think?
> >> >
> >> > Cheers,
> >> > Till
> >> >
> >> > On Mon, Oct 8, 2018 at 11:13 AM Chesnay Schepler <chesnay@apache.org
> >> > <mailto:chesnay@apache.org>> wrote:
> >> >
> >> >     I don't believe that to be the consensus. For starters it is
> >> >     contradictory; we can't /drop /flink-storm yet still /keep //some
> >> >     parts/.
> >> >
> >> >     From my understanding we drop flink-storm completely, and put a
> >> >     note in the docs that the bolt/spout wrappers of previous versions
> >> >     will continue to work.
> >> >
> >> >     On 08.10.2018 11:04, Till Rohrmann wrote:
> >> >>     Thanks for opening the issue Chesnay. I think the overall
> >> >>     consensus is to drop flink-storm and only keep the Bolt and Spout
> >> >>     wrappers. Thanks for your feedback!
> >> >>
> >> >>     Cheers,
> >> >>     Till
> >> >>
> >> >>     On Mon, Oct 8, 2018 at 9:37 AM Chesnay Schepler
> >> >>     <chesnay@apache.org <mailto:chesnay@apache.org>> wrote:
> >> >>
> >> >>         I've created
> >> >>         https://issues.apache.org/jira/browse/FLINK-10509 for
> >> >>         removing flink-storm.
> >> >>
> >> >>         On 28.09.2018 15:22, Till Rohrmann wrote:
> >> >>         > Hi everyone,
> >> >>         >
> >> >>         > I would like to discuss how to proceed with Flink's storm
> >> >>         compatibility
> >> >>         > layer flink-strom.
> >> >>         >
> >> >>         > While working on removing Flink's legacy mode, I noticed
> >> >>         that some parts of
> >> >>         > flink-storm rely on the legacy Flink client. In fact,
at
> >> >>         the moment
> >> >>         > flink-storm does not work together with Flink's new
> >> distributed
> >> >>         > architecture.
> >> >>         >
> >> >>         > I'm also wondering how many people are actually using
> >> >>         Flink's Storm
> >> >>         > compatibility layer and whether it would be worth porting
it.
> >> >>         >
> >> >>         > I see two options how to proceed:
> >> >>         >
> >> >>         > 1) Commit to maintain flink-storm and port it to Flink's
> >> >>         new architecture
> >> >>         > 2) Drop flink-storm
> >> >>         >
> >> >>         > I doubt that we can contribute it to Apache Bahir [1],
> >> >>         because once we
> >> >>         > remove the legacy mode, this module will no longer work
> >> >>         with all newer
> >> >>         > Flink versions.
> >> >>         >
> >> >>         > Therefore, I would like to hear your opinion on this and
in
> >> >>         particular if
> >> >>         > you are using or planning to use flink-storm in the future.
> >> >>         >
> >> >>         > [1] https://github.com/apache/bahir-flink
> >> >>         >
> >> >>         > Cheers,
> >> >>         > Till
> >> >>         >
> >> >>
> >> >
> >>
> >>

Mime
View raw message