flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Incremental updates
Date Wed, 25 May 2016 09:32:39 GMT
Hi Gosia,
right now, Flink is not doing incremental checkpoints. Every checkpoint is
fully valid in isolation. Incremental checkpointing came up several times
on ML discussions and we a planning to work on it once someone finds some
free time.

Cheers,
Aljoscha

On Wed, 25 May 2016 at 09:29 Rubén Casado <ruben.casado@treelogic.com>
wrote:

> Hi Gosia
>
> You can have a look to the PROTEUS project we are doing [1]. We are
> implementing incremental version of analytics operations. For example you
> can see in [2] the implementation of the incremental AVG. Maybe the code
> can give you some ideas :-)
>
>
> [1] https://github.com/proteus-h2020/proteus-backend/tree/development
> [2]
> https://github.com/proteus-h2020/proteus-backend/blob/development/src/main/java/com/treelogic/proteus/flink/incops/IncrementalAverage.java
>
> ______________________________________
>
> *Dr. Rubén Casado*
> Head of Big Data
> Treelogic
> <http://es.linkedin.com/in/rcasadot/> <https://twitter.com/ruben_casado>
> *ruben.casado.treelogic*
>
> +34 902 286 386 - +34 607 18 28 06
> Parque Tecnológico de Asturias · Parcela 30
> E33428 Llanera · Asturias [Spain]
> <http://www.treelogic.com>www.treelogic.com
> ______________________________________
>
>
> ----- Mensaje original -----
> De: "Malgorzata Kudelska" <m.j.kudelska@gmail.com>
> Para: user@flink.apache.org
> Enviados: Martes, 24 de Mayo 2016 22:01:28 GMT +01:00 Amsterdam / Berlín /
> Berna / Roma / Estocolmo / Viena
> Asunto: Incremental updates
>
>
> Hi,
> I have the following question. Does Flink support incremental updates?
>
> In particular, I have a custom StateValue object and during the
> checkpoints I would like to save only the fields that changed since the
> previous checkpoint. Is that possible?
>
> Regards,
> Gosia
>

Mime
View raw message