flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Hogan <c...@greghogan.com>
Subject Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the Flink 1.4.0 release
Date Thu, 17 Aug 2017 11:29:18 GMT
There’s an argument for delaying this change to 1.5 since the feature freeze is two weeks
away. There is little time to realize benefits from removing this code.

"The reason for that is that there is a lot of code mapping between the completely different
legacy format (1.1.x, not re-scalable) and the key-group-oriented format (1.2.x onwards, re-scalable).
It would greatly help the development of state and checkpointing features to drop that old
code.”

Greg


> On Aug 17, 2017, at 5:36 AM, Stefan Richter <s.richter@data-artisans.com> wrote:
> 
> One more comment about the consequences of this PR, as pointed out in the comments on
Github: this will also break direct compatibility for the CEP library between Flink 1.2 and
1.4. There is still a way to migrate via Flink 1.3: Flink 1.1/2 -> savepoint -> Flink
1.3 -> savepoint -> Flink 1.4.
> 
>> Am 16.08.2017 um 17:31 schrieb Stefan Richter <s.richter@data-artisans.com>:
>> 
>> Hi,
>> 
>> after there have been no objections since a long time, I took the next step and created
a PR that implements this change in commit 95e44099784c9deaf2ca422b8dfc11c3d67d7f82 of https://github.com/apache/flink/pull/4550
<https://github.com/apache/flink/pull/4550> . Announcing this here as a last opportunity
for further discussions. FYI, this will decrease the code base by almost 12K LOC. 
>> 
>> Best,
>> Stefan
>> 
>> 
>>> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <k.kloudas@data-artisans.com
<mailto:k.kloudas@data-artisans.com>>:
>>> 
>>> +1
>>> 
>>>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <trohrmann@apache.org <mailto:trohrmann@apache.org>>
wrote:
>>>> 
>>>> +1
>>>> 
>>>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s.richter@data-artisans.com
<mailto:s.richter@data-artisans.com>>
>>>> wrote:
>>>> 
>>>>> +1
>>>>> 
>>>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <sewen@apache.org <mailto:sewen@apache.org>>:
>>>>> 
>>>>> Seems like no one raised a concern so far about dropping the savepoint
>>>>> format compatibility for 1.1 in 1.4.
>>>>> 
>>>>> Leaving this thread open for some more days, but from the sentiment,
it
>>>>> seems like we should go ahead?
>>>>> 
>>>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org <mailto:sewen@apache.org>>
wrote:
>>>>> 
>>>>>> Hi users!
>>>>>> 
>>>>>> Flink currently maintains backwards compatibility for savepoint formats,
>>>>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x
can be
>>>>>> resumed in Flink 1.3.x
>>>>>> 
>>>>>> We are discussing how many versions back to support. The proposition
is
>>>>>> the following:
>>>>>> 
>>>>>> *   Suggestion: Flink 1.4.0 will be able to resume savepoints taken
with
>>>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and
1.0.x*
>>>>>> 
>>>>>> 
>>>>>> The reason for that is that there is a lot of code mapping between
the
>>>>>> completely different legacy format (1.1.x, not re-scalable) and the
>>>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would
greatly
>>>>>> help the development of state and checkpointing features to drop
that old
>>>>>> code.
>>>>>> 
>>>>>> Please let us know if you have concerns about that.
>>>>>> 
>>>>>> Best,
>>>>>> Stephan
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>> 
>> 
> 


Mime
View raw message