flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Kim <david....@braintreepayments.com>
Subject Re: Logging Exceptions
Date Wed, 25 May 2016 14:00:43 GMT
Awesome, thank you!

David

On Wed, May 25, 2016 at 4:54 AM Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi David,
> you are right, for some exceptions Flink only forwards to the
> web-dashboard/application client but does not print to the log file. I
> opened a Jira issue to track this:  FLINK-3969
> <https://issues.apache.org/jira/browse/FLINK-3969>.
>
> Thanks for reporting!
>
> Aljoscha
>
> On Mon, 23 May 2016 at 22:08 David Kim <david.kim@braintreepayments.com>
> wrote:
>
>> Hi Max!
>>
>> Unfortunately, that's not the behavior I'm seeing.
>>
>> I verified my log4.properties is configured properly because I do see
>> messages in the /log directory.
>>
>> However, for this stack trace (grabbed from the web dashboard), I do not
>> see it in my log file:
>>
>> java.lang.RuntimeException: Could not forward element to next operator
>> 	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:354)
>> 	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:337)
>> 	at org.apache.flink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:158)
>> 	at org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:127)
>> 	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
>> 	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
>> 	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
>> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>> 	at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Here's my sample program:
>>
>>
>> val env = StreamExecutionEnvironment.getExecutionEnvironment
>> env
>>   .fromCollection(List(1))
>>   .filter(element => {
>>     throw new RuntimeException("Throwing a runtime to test exception logging :)")
>>   })
>> env.execute("MyTestJobName")
>>
>>
>> The error is thrown in Task.java[1] and it was hard to track where flink would actually
log to the log file.
>>
>>
>> [1] https://github.com/apache/flink/blob/release-1.0.2-rc3/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java#L559
>>
>>
>> Thanks!
>>
>> David
>>
>>
>> On Mon, May 23, 2016 at 12:01 PM Maximilian Michels <mxm@apache.org>
>> wrote:
>>
>>> Hi David,
>>>
>>> I'm afraid Flink logs all exceptions. You'll find the exceptions in the
>>> /log directory.
>>>
>>> Cheers,
>>> Max
>>>
>>> On Mon, May 23, 2016 at 6:18 PM, David Kim <
>>> david.kim@braintreepayments.com> wrote:
>>>
>>>> Hello!
>>>>
>>>> Just wanted to check up on this. :)
>>>>
>>>> I grepped around for `log.error` and it *seems* that currently the
>>>> only events for logging out exceptions are for non-application related
>>>> errors.
>>>>
>>>> Thanks!
>>>> David
>>>>
>>>> On Fri, May 20, 2016 at 12:35 PM David Kim <
>>>> david.kim@braintreepayments.com> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> Using flink 1.0.2, I noticed that exceptions thrown during a flink
>>>>> program would show up on the flink dashboard in the 'Exceptions' tab.
>>>>> That's great!
>>>>>
>>>>> However, I don't think flink currently logs this same exception. I was
>>>>> hoping there would be an equivalent `log.error` call so that third party
>>>>> logging frameworks can also act upon such errors.
>>>>>
>>>>> If this currently the known behavior, would it be troublesome to also
>>>>> make a `log.error` call around the code that is responsible for sending
the
>>>>> exception to the dashboard?
>>>>>
>>>>> If there's a misconfiguration on my end, let me know!
>>>>>
>>>>> Thanks!
>>>>> David
>>>>>
>>>>
>>>

Mime
View raw message