flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject Re: Exceptions from collector.collect after cancelling job
Date Fri, 30 Sep 2016 16:09:33 GMT
My flat map function is catching & logging the exception. The try block happens to encompass
the call to Collector#collect().

I will move the call to collect outside of the try. That should silence the log message.

On 9/30/16, 3:51 AM, "Ufuk Celebi" <uce@apache.org> wrote:

>On Thu, Sep 29, 2016 at 9:29 PM, Shannon Carey <scarey@expedia.com> wrote:
>> It looks like Flink is disabling the objects that the FlatMap collector
>> relies on before disabling the operator itself. Is that expected/normal? Is
>> there anything I should change in my FlatMap function or job code to account
>> for it?
>Hey Shannon,
>Flink actually does cancel the tasks *before* cleaning up the network
>resources that throw the root Exception here.
>We actually don't log any Exceptions that are thrown during
>cancellation, because it is possible that the user code/operator use
>the closed resources concurrently with cancellation (your stack traces
>essentially), but it looks like in some places we don't respect this.
>Can you tell which classes actually log this? Would be good to fix
>this if possible as it is very confusing and looks quite bad. I don't
>expect it to be an actual problem though.
>– Ufuk
View raw message