flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zhangminglei <18717838...@163.com>
Subject Re: [DISCUSS] Flink 1.6 features
Date Mon, 25 Jun 2018 02:45:18 GMT
Hi, Community

By the way, there is a very important feature I think it should be. Currently, the BucketingSink
does not support when a bucket is ready for user use. This situation will be very obvious
when flink work with offline end. We called that real time/offline integration in business.
In this case, we should let the user can do some extra work when the bucket is ready. And
here is the JIRA for this https://issues.apache.org/jira/browse/FLINK-9609 <https://issues.apache.org/jira/browse/FLINK-9609>


> 在 2018年6月4日,下午5:21,Stephan Ewen <sewen@apache.org> 写道:
> Hi Flink Community!
> The release of Apache Flink 1.5 has happened (yay!) - so it is a good time to start talking
about what to do for release 1.6.
> == Suggested release timeline ==
> I would propose to release around end of July (that is 8-9 weeks from now).
> The rational behind that: There was a lot of effort in release testing automation (end-to-end
tests, scripted stress tests) as part of release 1.5. You may have noticed the big set of
new modules under "flink-end-to-end-tests" in the Flink repository. It delayed the 1.5 release
a bit, and needs to continue as part of the coming release cycle, but should help make releasing
more lightweight from now on.
> (Side note: There are also some nightly stress tests that we created and run at data
Artisans, and where we are looking whether and in which way it would make sense to contribute
them to Flink.)
> == Features and focus areas ==
> We had a lot of big and heavy features in Flink 1.5, with FLIP-6, the new network stack,
recovery, SQL joins and client, ... Following something like a "tick-tock-model", I would
suggest to focus the next release more on integrations, tooling, and reducing user friction.

> Of course, this does not mean that no other pull request gets reviewed, an no other topic
will be examined - it is simply meant as a help to understand where to expect more activity
during the next release cycle. Note that these are really the coarse focus areas - don't read
this as a comprehensive list.
> This list is my first suggestion, based on discussions with committers, users, and mailing
list questions.
>   - Support Java 9 and Scala 2.12
>   - Smoothen the integration in Container environment, like "Flink as a Library", and
easier integration with Kubernetes services and other proxies.
>   - Polish the remaing parts of the FLIP-6 rewrite
>   - Improve state backends with asynchronous timer snapshots, efficient timer deletes,
state TTL, and broadcast state support in RocksDB.
>   - Extends Streaming Sinks:
>      - Bucketing Sink should support S3 properly (compensate for eventual consistency),
work with Flink's shaded S3 file systems, and efficiently support formats that compress/index
arcoss individual rows (Parquet, ORC, ...)
>      - Support ElasticSearch's new REST API
>   - Smoothen State Evolution to support type conversion on snapshot restore
>   - Enhance Stream SQL and CEP
>      - Add support for "update by key" Table Sources
>      - Add more table sources and sinks (Kafka, Kinesis, Files, K/V stores)
>      - Expand SQL client
>      - Integrate CEP and SQL, through MATCH_RECOGNIZE clause
>      - Improve CEP Performance of SharedBuffer on RocksDB

View raw message