flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Janardhan Reddy <janardhan.re...@olacabs.com>
Subject Re: Way to hold execution of one of the map operator in Co-FlatMaps
Date Mon, 27 Jun 2016 09:14:09 GMT
Hi,

Instead of buffering can we use event creation time and watermarks ?

On Mon, Jun 27, 2016 at 2:32 PM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi,
> the two map functions are called by the same thread, so waiting in one
> function would block all processing. What you could do is buffer elements
> from one input and only process them when an element arrives on the other
> input.
>
> Cheers,
> Aljoscha
>
> On Sun, 26 Jun 2016 at 13:36 Biplob Biswas <revolutionisme@gmail.com>
> wrote:
>
>> Hi,
>>
>> I was wondering whether it is possible to stop streaming data in from one
>> of
>> the map operators until some data arrives in the second map operator.
>>
>> For ex,
>>
>> if i have ds1.connect(ds2).map(new coflatmapper())
>>
>> then, i need data to stop flowing from ds1 until some data arrives in ds2.
>>
>> Is that possible? I know that's a problematic situation as ds1 may get
>> overwhelmed with data if not processed quickly, but any work around would
>> really helpful.
>>
>>
>> I was thinking of busy waiting, but I doubt that, the 2 maps work in
>> different threads.
>>
>> Regards
>> Biplob
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Way-to-hold-execution-of-one-of-the-map-operator-in-Co-FlatMaps-tp7689.html
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive at Nabble.com.
>>
>

Mime
View raw message