flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: ListState<T> - elements order
Date Mon, 17 Sep 2018 14:37:34 GMT
Hi all,

Flink does not provide any guarantees about the order of the elements in a list and it leaves
it to the state-backends.

This means that semantics between different backends may differ, and even if something holds
now for one of them, if 
RocksDB or a filesystem decides to change its semantics, then any assumptions will not hold
anymore.

So I would recommend to program under the assumption that no order guarantees are provided.

Cheers,
Kostas

> On Sep 14, 2018, at 4:30 AM, vino yang <yanghua1127@gmail.com> wrote:
> 
> Hi Aljoscha,
> 
> Regarding window merging, as you said, it's not clear, because Flink does some internal
work. 
> But if it's just for the user, isn't it clear without any internal operations? I think
if the user explicitly uses it, it should conform to the basic List semantics. Otherwise why
define it instead of using SetListState directly?
> 
> Thanks, vino.
> 
> Aljoscha Krettek <aljoscha@apache.org <mailto:aljoscha@apache.org>> 于2018年9月13日周四
下午10:42写道:
> Hi,
> 
> this is not clearly defined anywhere, and I was always working under the assumption that
the order is not preserved. This potentially allows more optimizations by the system, and
for example in case of merging windows we don't know the order of elements in a ListState
after a merge.
> 
> Best,
> Aljoscha
> 
>> On 6. Sep 2018, at 08:19, vino yang <yanghua1127@gmail.com <mailto:yanghua1127@gmail.com>>
wrote:
>> 
>> Hi Alexey,
>> 
>> The answer is Yes, which preserves the semantics of the List's order of elements.
>> 
>> Thank, vino.
>> 
>> Alexey Trenikhun <yender@msn.com <mailto:yender@msn.com>> 于2018年9月6日周四
上午10:55写道:
>> Hello,
>> Does keyed managed ListState<T> preserve elements order, for example if I call
listState.add(e1); listState.add(e2); listState.add(e3); , does ListState<T> guarantee
that listState.get() will return elements in order they were added (e1, e2, e3)
>> 
>> Alexey
>> 
>> 
> 


Mime
View raw message