Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <3dd83d0b-3d2d-e4f8-2f82-bc4a766442f2@apache.org>
References: <etPan.595bd7b1.3da898fe.118@apache.org> <3dd83d0b-3d2d-e4f8-2f82-bc4a766442f2@apache.org>
From: SHI Xiaogang <shixiaogangg@gmail.com>
Date: Wed, 5 Jul 2017 10:36:02 +0800
Message-ID: <CAHM4ZtkUswEx10fN5syiL=1Z+uVDFv36ZB8cpjGk4ii+rUhH0g@mail.gmail.com>
Subject: Re: [DISCUSS] FLIP-22: Eager State Declaration
To: dev@flink.apache.org
Content-Type: multipart/alternative; boundary="001a114984147ec5ee055388dcf7"
archived-at: Wed, 05 Jul 2017 02:36:09 -0000

--001a114984147ec5ee055388dcf7
Content-Type: text/plain; charset="UTF-8"

Hi Tzu-Li,

Thanks for the proposal. The changes are great. I have several questions
about some details.

First, do you have any plan to provide a method to remove states? Now
states can only be created (either lazily or eagerly), but cannot be
removed. We cannot remove those states not registered because they may be
accessed later (with those deprecated methods).

Second, what about exposing namespaces to users? Now namespaces are only
used in window streams and all user states are in the void namespace. But
some users may come across similar scenarios to window streams where states
are closely related to arrived records and cannot be known beforehand.
Since namespaces are not exposed, they have to create new states when new
records arrive. MapState is another choice, but will be less efficient in
some cases. If we can expose namespaces to users, these users may benefit
from eagerly declared states. I think the change will not break existing
interfaces.

Looking forwards to your comments.

Regards,
Xiaogang


2017-07-05 3:53 GMT+08:00 Chesnay Schepler <chesnay@apache.org>:

> Could you add an example to the FLIP for how a user can register a state
> with the methods in the RichFunction interface?
> Currently it only contains an example for the annotation option.
>
> These methods look like they are called by the user, but that doesn't
> really make sense to me as after all the user has to
> implement them.
>
> To me a more intuitive signature would be
>
> |void registerKeyedState(StateDescriptorRegistry registry);|
>
> that is called by the system when a UDF is provided by a user who then
> registers all the state descriptors he has.
>
>
> On 04.07.2017 20:00, Tzu-Li (Gordon) Tai wrote:
>
>> Hi Flink devs!
>>
>> I would like to propose the following FLIP - Eager State Declaration for
>> Flink managed state: https://cwiki.apache.org/confl
>> uence/display/FLINK/FLIP-22%3A+Eager+State+Declaration.
>> The proposal is a result of some offline discussions with Aljoscha
>> Krettek, Stephan Ewen, and Stefan Richter.
>>
>> With how the current managed state declaration interfaces work, users may
>> declare state lazily while jobs are running.
>> This behavior is a direct blocker for several state management features
>> we wish to make a reality in the future.
>> I also see it as an opportunity to make the interfaces for keyed /
>> operator managed state declarations more unified at the API level, as well
>> as improved user experience for general use cases.
>>
>> The most important part of the required changes is the deprecation of
>> existing APIs and introducing new state declaration interfaces.
>> Since this would be a rework of the state interfaces, it would be great
>> to hear thoughts on this and make sure that the proposal is what we want in
>> the long run!
>>
>> Happy to hear feedback on this :)
>>
>> Cheers,
>> Gordon
>>
>
>
>

--001a114984147ec5ee055388dcf7--