flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Excessive stdout is causing java heap out of mem
Date Mon, 22 May 2017 19:04:46 GMT
Hi Fritz,

The TaskManagers are not buffering all stdout for the webinterface (at
least I'm not aware of that). Did the error occur when accessing the log
from the JobManager?
Flinks web front end lazily loads the logs from the taskmanagers.

The suggested method for logging is to use slf4j for logging, so the
following code snippets :

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

private static final Logger LOG = LoggerFactory.getLogger(MyJob.class);

Then you can do stuff like:

LOG.info("My log statement");

Also, using a logging Framework will allow you to redirect the log contents
of your job to a separate file.

But I'm not sure if the logging is really causing the TaskManager JVMs to
die ...

On Sat, May 20, 2017 at 3:12 AM, Fritz Budiyanto <fbudiyan@icloud.com>

> Hi,
> I notice that when I enabled DataStreamSink’s print() for debugging,
> (kinda excessive printing), its causing java Heap out of memory.
> Possibly the Task Manager is buffering all stdout for the WebInterface? I
> haven’t spent time debugging it, but I wonder if this is expected where
> massive print will exhaust java heap, and I’m using standalone mode.
> Is there a way to disable this memory logging for web interface, and just
> redirect stdout to file instead with file rotation?
> What is the suggested method of logging ?
> —
> Fritz

View raw message