flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Dripping the Flink-on-Tez code for Flink 1.0
Date Fri, 08 Jan 2016 15:43:31 GMT
Hi all!

Currently, Flink has a module to run batch program code on Tez rather than
Flink's own distributed execution engine.

I would suggest that we drop this code for the next release (1.0) as part
of a code consolidation:

  - There seems little in both the Flink and the Tez community to use and
expand this functionality.

  - The original motivation (better exploit resource elasticity in YARN)
will no longer be valid in the near future. I am re-working the YARN
integration currently to make it more elastic and make it possible to run
Flink on Mesos.

  - The Flink-on-Tez code is rather POC status. Large scale testing it and
making adding all missing features will take more effort than making
Flink's own YARN integration resource elastic.


Please let me know what you think!
Especially @Kostas, since you wrote the initial POC, I'd be interested in
your opinion.

Greetings,
Stephan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message