flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Getting JobManager address and port within a running job
Date Wed, 09 Aug 2017 15:18:04 GMT
Ok, I saw that one. Unfortunately it's still not clear to me how that would work. Could you
maybe highlight an actual flow of events and describe what will (or should) happen? Describe
the current state of the system, i.e. what state is there, then describe what happens when
events with given keys come in, how they affect the state and what output should be produced.

Best,
Aljoscha
> On 9. Aug 2017, at 15:38, Biplob Biswas <revolutionisme@gmail.com> wrote:
> 
> Hi Aljoscha,
> 
> Thanks for the link. I read through it but I still can't imagine
> implementing something similar for my usecase. 
> 
> I explained my usecase to Fabian in a previous post, I would try to be again
> as clear as possible. 
> So, my use case, in short, is something like below: 
> 
> 1) From input stream of events, create 2 different local state store (one
> with key as event_id and the other with key as derived_key), where both have
> 1-N relationship. For eg.  1 -> [A,B,C] and A -> [1,2,3] 
> 2) After we have both the state, traverse through the events a second time,
> generate the derived_keys and based on these keys query state B to get a
> list of event_ids which in turn is used to query the state A to fetch the
> events for these ids
> 3) Do this only till a depth of 1, i.e. stop after doing the above process
> just once and don't traverse back
> 4) Combine and merge all the events which were fetched because they are
> related
> 
> So, I can't imagine how I can use the scatter->state->gather pattern to
> achieve the desired result.Maybe I am wrong and I have missed something, so
> any insights would be useful! 
> 
> 
> 
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Getting-JobManager-address-and-port-within-a-running-job-tp14656p14770.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.


Mime
View raw message