flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Yao <g...@data-artisans.com>
Subject Re: Access logs for a running Flink app in YARN cluster
Date Fri, 13 Apr 2018 19:13:35 GMT

I see two options:

1. You can login to the slave machines, which run the NodeManagers, and
the container logs. The path of the container logs can be configured in
yarn-site.xml with the key yarn.nodemanager.log-dirs. In my tests with EMR,
logs are stored at /var/log/hadoop-yarn/containers/ by default. For example:

  sudo ls
  taskmanager.err  taskmanager.log  taskmanager.out

2. You can download the logs via HTTP from Flink:


To get a list of taskmanagers:



On Fri, Apr 13, 2018 at 2:09 AM, Tao Xia <tao@udacity.com> wrote:

> Any good way to get access container logs from a running Flink app in YARN
> cluster in EMR?
> You can view the logs through YARN UI. But cannot programmatically access
> it and send to other services.
> The log aggregator only runs when the application finishes or a minimum
> 3600 secs copy. Any way we can get the logs more frequently?

View raw message