Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 757E9200C8C for ; Tue, 6 Jun 2017 19:34:41 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 73E8C160BD3; Tue, 6 Jun 2017 17:34:41 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 6C620160BB7 for ; Tue, 6 Jun 2017 19:34:40 +0200 (CEST) Received: (qmail 83895 invoked by uid 500); 6 Jun 2017 17:34:39 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 83885 invoked by uid 99); 6 Jun 2017 17:34:39 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jun 2017 17:34:39 +0000 Received: from [192.168.0.6] (ip5f5af52d.dynamic.kabel-deutschland.de [95.90.245.45]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 528CC1A003E; Tue, 6 Jun 2017 17:34:38 +0000 (UTC) From: Aljoscha Krettek Message-Id: <32432824-C36D-44F6-ADFE-B0EADE0113E0@apache.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_94D25262-D8B4-4280-BC84-8F9E21AE3090" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: AsyncCollector Does not release the thread (1.2.1) Date: Tue, 6 Jun 2017 19:34:35 +0200 In-Reply-To: Cc: user@flink.apache.org To: Steve Robert References: <802F130F-DD15-4356-B4FF-83DD54FD10C0@apache.org> X-Mailer: Apple Mail (2.3273) archived-at: Tue, 06 Jun 2017 17:34:41 -0000 --Apple-Mail=_94D25262-D8B4-4280-BC84-8F9E21AE3090 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Ok, thanks for letting us know. I=E2=80=99ll investigate. > On 6. Jun 2017, at 19:28, Steve Robert wrote: >=20 > Hi Aljoscha , >=20 > thank you for your reply, > yes the queue being filled up and no more elements are being = processed.(In relation to the limit defined at the "orderedWait" = function call). > To add additional information, if I run the test on a local cluster I = can see that the job never ends because the AsyncFunction stay blocked = As if there was no call to the "collect" method > Best, > Steve >=20 > On Tue, Jun 6, 2017 at 4:56 PM, Aljoscha Krettek > wrote: > Hi, >=20 > As far as I know calling collect(Throwable) should also finish the = promise that would otherwise fulfilled by successfully collecting a = result. If not then you might have found a bug. What makes you think = that the Thread is not being released? Is your queue being filled up and = no more elements are being processed? >=20 > Regarding your other question, yes, you can collect an empty = Collection for signalling that there was no result. >=20 > Best, > Aljoscha >=20 >> On 8. May 2017, at 21:47, Steve Robert > wrote: >>=20 >> Hi guys,=20 >>=20 >> AsyncCollector.collect(Throwable) method seem to not release the = Thread. >> This scenario may be problematic when calling an external API >> In the case of a timeout error there is no data to collect. >>=20 >> for example : >>=20 >> CompletableFuture.supplyAsync(() -> asyncCallTask(input)) >> .thenAccept((Collection> result) -> { >>=20 >> this.tupleEmited.getAndIncrement(); >>=20 >> asyncCollector.collect(result); >> }) >> .exceptionally((ex) -> { >> asyncCollector.collect(ex); >> return null; >> }); >> } >> it is possible to create an empty Collection and collect this empty = collection to force the Thread to be released but this workflow seems = strange to me. >> thank for your help >> =20 >>=20 >> --=20 >> Steve Robert >> Software Engineer >> srobert@qualys.com >> T <> >> Qualys, Inc. =E2=80=93 Continuous Security >> Blog | Community = | Twitter >> >=20 >=20 >=20 > --=20 > Steve Robert > Software Engineer > srobert@qualys.com > T <> > Qualys, Inc. =E2=80=93 Continuous Security > Blog | Community = | Twitter > --Apple-Mail=_94D25262-D8B4-4280-BC84-8F9E21AE3090 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Ok, thanks for letting us know. I=E2=80=99ll investigate.
On 6. Jun 2017, at 19:28, Steve Robert <srobert@qualys.com> = wrote:

Hi Aljoscha ,

thank you for your reply,
 yes the queue being filled up and no more elements are = being processed.(In relation to the limit defined at the "orderedWait" function call).
To = add additional information, if I run the test on a local cluster I can = see that the job never ends because the AsyncFunction stay = blocked As if there was no call to  the "collect" = method
Best,
Steve

On Tue, Jun 6, 2017 at 4:56 PM, Aljoscha Krettek = <aljoscha@apache.org> = wrote:
Hi,

As = far as I know calling collect(Throwable) should also finish the promise = that would otherwise fulfilled by successfully collecting a result. If = not then you might have found a bug. What makes you think that the = Thread is not being released? Is your queue being filled up and no more = elements are being processed?

Regarding your other question, yes, you = can collect an empty Collection for signalling that there was no = result.

Best,
Aljoscha

On 8. = May 2017, at 21:47, Steve Robert <srobert@qualys.com> wrote:

Hi guys, 

AsyncCollector.collect(Throwable) method  seem to  not release  the = Thread.
This scenario may be problematic when = calling an external API
In the case = of a timeout error there is no data to collect.

for example :

  CompletableFuture.supplyAsync(() -> =
asyncCallTask(input))
            =
.thenAccept((Collection<Tuple3<String, streamDTO, =
Integer>> result) -> {

            =
    this.tupleEmited.getAndIncrement();

                asyncCollector.collect(result);
            })
            .exceptionally((ex) =
-> {
                asyncCollector.collect(ex);
             =
   return =
null;
            });
}
it = is possible to create an empty Collection and collect this empty = collection to force the Thread to be released but this workflow seems = strange to me.
thank for your help
 

-- =
Steve Robert
Software Engineer
srobert@qualys.com
T
Qualys, Inc. =E2=80=93 Continuous = Security
Blog |=20 Community |=20 Twitter




--
Steve Robert
Software Engineer
srobert@qualys.com
T
Qualys, Inc. =E2=80=93 Continuous = Security
Blog |=20 Community |=20 Twitter

= --Apple-Mail=_94D25262-D8B4-4280-BC84-8F9E21AE3090--