flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <kklou...@gmail.com>
Subject Re: Important (proposed) CEP changes for Flink 1.5.
Date Wed, 21 Feb 2018 16:48:03 GMT
Hi Vishal,

Yes these changes will only affect CEP.

Cheers,
Kostas

> On Feb 21, 2018, at 5:42 PM, Vishal Santoshi <vishal.santoshi@gmail.com> wrote:
> 
> That should be fine by us. We do not plan to do a SP and restore on our prod CEP pipe,
when we move to 1.5 from 1.4. I do hope that all other pipes will remain back ward compatible
( 1.4 to 1.5 ).
> 
> On Wed, Feb 21, 2018 at 5:06 AM, Kostas Kloudas <kkloudas@gmail.com <mailto:kkloudas@gmail.com>>
wrote:
> Hi all,
> 
> Currently due to backwards compatibility there are some issues that seem to be affecting
CEP users that use RocksDB. As an example you can see this:
> 
> https://issues.apache.org/jira/browse/FLINK-7756 <https://issues.apache.org/jira/browse/FLINK-7756>
> 
> We suspect that the issues have mainly to do with the way state was kept internally in
previous versions (using java serialization) and got inherited to the newer ones.
> 
> Given this, we would like to introduce a revamped state serialization strategy which
solves the sources of these problems and also reduces the size of the state kept at each checkpoint.
The problem with this, is that the state format changes so already existing jobs will not
be able to restart from a savepoint taken from a previous version of the library.
> 
> I am writing this email to see if there are any objections to merging these changes in
the 1.5 release which is about to be released. I suggest to wait for responses until Friday
5pm UTC+1, and if there is non, then we merge. If there are any, then we have to see how to
proceed.
> 
> You can try out the changes in this branch: https://github.com/aljoscha/flink/commits/fix-flink-cep-serialization
<https://github.com/aljoscha/flink/commits/fix-flink-cep-serialization> and report here
if you have any problems or suggestions.
> 
> Also feel free to forward this to other users, if I forgot someone.
> 
> Cheers,
> Kostas
> 
> 


Mime
View raw message