flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Flink QueryableState with Sliding Window on RocksDB
Date Mon, 31 Jul 2017 19:30:42 GMT
Having an operator that updates state from one stream and queries it to
process the other stream is actually a common pattern.
As I said, I don't know your use case but I don't think that a
CoProcessFunction would result in a mess.

QueryableState will have quite a bit of overhead because the request and
response will always go over the network.

2017-07-31 15:23 GMT+02:00 Biplob Biswas <revolutionisme@gmail.com>:

> Hi Fabian,
> Thanks for the insight, I am currently exploring  QueryableStateClient and
> would attempt to get the value for a corresponding key using the
> getkvstate() function,
> I was confused about the jobId but I am expecting this would provide me
> with
> the jobid of the current job -
> ExecutionEnvironment.getExecutionEnvironment().getId()
> I thought about updating and doing the look up from a single operator but
> then I believe my code would be a mess with a lot of logic and no logical
> separation, so that's the last thing I want to try.
> My team is also exploring KStreams with Ktables and they claim that does
> what we want to do, will update with any further information. If possible
> contribute to Flink to use the keyedstate natively in the same job as well.
> Thanks for the help,
> Biplob
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Flink-
> QueryableState-with-Sliding-Window-on-RocksDB-tp14514p14558.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.

View raw message