flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the Flink 1.4.0 release
Date Thu, 17 Aug 2017 11:34:57 GMT
I think we are still doing changes for which this is relevant. Also I cannot really see a benefit
in delaying this because the whole discussion will apply in exactly the same way to 1.5.

> Am 17.08.2017 um 13:29 schrieb Greg Hogan <code@greghogan.com>:
> 
> There’s an argument for delaying this change to 1.5 since the feature freeze is two
weeks away. There is little time to realize benefits from removing this code.
> 
> "The reason for that is that there is a lot of code mapping between the completely different
legacy format (1.1.x, not re-scalable) and the key-group-oriented format (1.2.x onwards, re-scalable).
It would greatly help the development of state and checkpointing features to drop that old
code.”
> 
> Greg
> 
> 
>> On Aug 17, 2017, at 5:36 AM, Stefan Richter <s.richter@data-artisans.com> wrote:
>> 
>> One more comment about the consequences of this PR, as pointed out in the comments
on Github: this will also break direct compatibility for the CEP library between Flink 1.2
and 1.4. There is still a way to migrate via Flink 1.3: Flink 1.1/2 -> savepoint ->
Flink 1.3 -> savepoint -> Flink 1.4.
>> 
>>> Am 16.08.2017 um 17:31 schrieb Stefan Richter <s.richter@data-artisans.com>:
>>> 
>>> Hi,
>>> 
>>> after there have been no objections since a long time, I took the next step and
created a PR that implements this change in commit 95e44099784c9deaf2ca422b8dfc11c3d67d7f82
of https://github.com/apache/flink/pull/4550 <https://github.com/apache/flink/pull/4550>
. Announcing this here as a last opportunity for further discussions. FYI, this will decrease
the code base by almost 12K LOC. 
>>> 
>>> Best,
>>> Stefan
>>> 
>>> 
>>>> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <k.kloudas@data-artisans.com
<mailto:k.kloudas@data-artisans.com>>:
>>>> 
>>>> +1
>>>> 
>>>>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <trohrmann@apache.org <mailto:trohrmann@apache.org>>
wrote:
>>>>> 
>>>>> +1
>>>>> 
>>>>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s.richter@data-artisans.com
<mailto:s.richter@data-artisans.com>>
>>>>> wrote:
>>>>> 
>>>>>> +1
>>>>>> 
>>>>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <sewen@apache.org
<mailto:sewen@apache.org>>:
>>>>>> 
>>>>>> Seems like no one raised a concern so far about dropping the savepoint
>>>>>> format compatibility for 1.1 in 1.4.
>>>>>> 
>>>>>> Leaving this thread open for some more days, but from the sentiment,
it
>>>>>> seems like we should go ahead?
>>>>>> 
>>>>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org
<mailto:sewen@apache.org>> wrote:
>>>>>> 
>>>>>>> Hi users!
>>>>>>> 
>>>>>>> Flink currently maintains backwards compatibility for savepoint
formats,
>>>>>>> which means that savepoints taken with Flink version 1.1.x and
1.2.x can be
>>>>>>> resumed in Flink 1.3.x
>>>>>>> 
>>>>>>> We are discussing how many versions back to support. The proposition
is
>>>>>>> the following:
>>>>>>> 
>>>>>>> *   Suggestion: Flink 1.4.0 will be able to resume savepoints
taken with
>>>>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x
and 1.0.x*
>>>>>>> 
>>>>>>> 
>>>>>>> The reason for that is that there is a lot of code mapping between
the
>>>>>>> completely different legacy format (1.1.x, not re-scalable) and
the
>>>>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would
greatly
>>>>>>> help the development of state and checkpointing features to drop
that old
>>>>>>> code.
>>>>>>> 
>>>>>>> Please let us know if you have concerns about that.
>>>>>>> 
>>>>>>> Best,
>>>>>>> Stephan
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>> 
>> 
> 


Mime
View raw message