flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject [DISCUSS] Towards a leaner flink-dist
Date Fri, 18 Jan 2019 09:59:14 GMT

the binary distribution that we release by now contains quite a lot of 
optional components, including various filesystems, metric reporters and 
libraries. Most users will only use a fraction of these, and as such 
pretty much only increase the size of flink-dist.

With Flink growing more and more in scope I don't believe it to be 
feasible to ship everything we have with every distribution, and instead 
suggest more of a "pick-what-you-need" model, where flink-dist is rather 
lean and additional components are downloaded separately and added by 
the user.

This would primarily affect the /opt directory, but could also be 
extended to cover flink-dist. For example, the yarn and mesos code could 
be spliced out into separate jars that could be added to lib manually.

Let me know what you think.



View raw message