flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudia Wegmann <c.wegm...@kasasi.de>
Subject partial savepoints/combining savepoints
Date Mon, 01 Aug 2016 13:49:49 GMT
Hey everyone,

I've got some questions regarding savepoints in Flink. I have the following situation:

There is a microservice that reads data from Kafka topics, creates Flink streams from this
data and does different computations/pattern matching workloads. If the overall workload for
this service becomes too big, I want to start a new instance of this service and share the
work between the running services. To accomplish that, I thought about using Flinks savepoint
mechanism. But there are some open questions:

1.)    Can I combine two or more savepoints in one program?
Think of two services already running. Now I'm starting up a third service. The new one would
get savepoints from the already running services. It than would continue computation of some
streams while the other services would discard calculation on these streams now calculated
by the new service. So, is it possible to combine two or more savepoints in one program?

2.)    Another approach I could think of for accomplishing the introduction of a new service
would be, to just take a savepoint of the streams that change service. Can I only take a savepoint
of a part of the running job?

Thanks for your comments and best wishes,

View raw message