flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@informatik.hu-berlin.de>
Subject Storm compatibility layer for Flink (first beta available)
Date Thu, 02 Apr 2015 17:31:24 GMT
Hi @all,

I started to work on an compatibility layer to run Storm Topologies on
Flink. I just pushed a first beta:
https://github.com/mjsax/flink/tree/flink-storm-compatibility

Please check it out, and let me know how you like it. In this first
version, I tried to code without changing too many things in the
existing code. However, a deeper integration would be nice.

There are many things that do not work yet. Currently, only shuffle and
fieldsGrouping is supported (and only Storm's default output stream).
Furthermore, topologies must be "simple", ie, they cannot be configured
with a Config object and Storm meta information (ie, TopologyContext) is
also not there.

The layer can be used to integrate existing Spouts and/or Bolts as Flink
operators into a regular Flink program. Furthermore, a whole Storm
topology can be executed by switching from TopologyBuilder to
FlinkTopologyBuilder and from LocalCluster/NimbusClient/StormSubmitter
to FlinkLocalCluster/FlinkClient/FlinkStormSubmitter.

Examples of both cases are given as ITCases.


Cheers,
  Matthias



Mime
View raw message