flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Logging in Flink 0.9.0-milestone-1
Date Fri, 26 Jun 2015 09:58:17 GMT
There are two different forms of progress logging:

 - The log4j framework logs everything (as configured in the
log4j.properties)
 - Additionally, the client sysout prints the progress reports

If you want the command line to be quiet, you need to disable sysout
logging on the execution environment for now.

We are thinking about adding a flag to the CLI frontend to make the console
quiet (no progress reports).

Greetings,
Stephan




On Fri, Jun 26, 2015 at 11:47 AM, Stefan Bunk <stefan.bunk@googlemail.com>
wrote:

> Hi Robert,
>
> this problem persists in the 0.9 release. Using `disableSysoutLogging()`
> works, but I'd rather configure this in the log4j.xml. Is this possible?
>
> Best,
> Stefan
>
> On 14 April 2015 at 20:55, Robert Metzger <rmetzger@apache.org> wrote:
>
>> You can control the logging behavior from the ExecutionConfig
>> (env.getExecutionConfig()).
>> There is a method (disableSysoutLogging()) that you can use. (In
>> 0.9-SNAPSHOT only).
>>
>> Sorry again that you ran into this issue.
>>
>> On Tue, Apr 14, 2015 at 8:45 PM, Robert Metzger <rmetzger@apache.org>
>> wrote:
>>
>>> Ah, I see.
>>>
>>> The issue is this line in the JobClient.scala here:
>>> https://github.com/apache/flink/blob/release-0.9.0-milestone-1-rc1/flink-runtime/src/main/scala/org/apache/flink/runtime/client/JobClient.scala#L97
>>>
>>> As you can see, its doing sysout logging.
>>> In the current master, this has been reworked:
>>> https://github.com/apache/flink/blob/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60
>>> (I'm currently looking up how to control the behavior)
>>>
>>> If its possible for you, you can switch to 0.9-SNAPSHOT.
>>>
>>> Best,
>>> Robert
>>>
>>>
>>> On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk <stefan.bunk@googlemail.com
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> I am also using IntelliJ, and I am starting directly from the IDE.
>>>> Local execution.
>>>> This is what my logging output looks like: [1].
>>>>
>>>> I am getting my logger via:
>>>> val log = org.apache.log4j.Logger.getLogger(getClass)
>>>>
>>>> [1] https://gist.github.com/knub/1c11683601b4eeb5d51b
>>>>
>>>> On 14 April 2015 at 18:47, Robert Metzger <rmetzger@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> how are you starting Flink? Out of the IDE? Using the scripts?
>>>>>
>>>>> I just created a new flink project with the milestone version. Just
>>>>> putting your log4j.xml into the resources folder enabled the logging
(I've
>>>>> set it to INFO for flink and it worked).
>>>>> I've used IntelliJ and started the WordCount.java from the quickstart.
>>>>>
>>>>> I think I need more details about your setup to reproduce the issue.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <
>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>
>>>>>> Hi Robert,
>>>>>>
>>>>>> thanks for the info.
>>>>>> Adding the parameter didn't help. My logging file is found and my
>>>>>> logging configuration for my own logging is working (even without
the
>>>>>> parameter), it's just that the file in the jar seems to be preferred
over
>>>>>> my file.
>>>>>>
>>>>>> Best,
>>>>>> Stefan
>>>>>>
>>>>>> On 14 April 2015 at 17:16, Robert Metzger <rmetzger@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Stefan,
>>>>>>>
>>>>>>> we made a stupid mistake in the 0.9.0-milestone-1 release by
>>>>>>> including our log4j.properties into the flink-runtime jar. its
also in the
>>>>>>> fat jar in flink-dist.
>>>>>>>
>>>>>>> Maybe you can pass the name of your log4j file to your application
>>>>>>> with -Dlog4j.configuration=log4j.xml?
>>>>>>>
>>>>>>> The issue is already fixed in the current master but that doesn't
>>>>>>> help 0.9.0-milestone-1 users :(
>>>>>>>
>>>>>>> Best,
>>>>>>> Robert
>>>>>>>
>>>>>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <
>>>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>>>
>>>>>>>> Hi Flinkers,
>>>>>>>>
>>>>>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's
logging
>>>>>>>> output again in my console (local execution).
>>>>>>>> I have a log4j.xml under src/main/resources, which says not
to log
>>>>>>>> any Flink job progress updates, and which worked fine so
far:
>>>>>>>>
>>>>>>>> [...]
>>>>>>>> <logger name="org.apache.flink">
>>>>>>>>                 <level value="ERROR" />
>>>>>>>>                 <appender-ref ref="Console" />
>>>>>>>> </logger>
>>>>>>>> [...]
>>>>>>>> See the full file at [1].
>>>>>>>>
>>>>>>>> However, this file seems to be ignored now. What do I need
to
>>>>>>>> change?
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Stefan
>>>>>>>>
>>>>>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message