fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Walch <mwa...@apache.org>
Subject Re: debugging fluo
Date Tue, 01 Nov 2016 18:36:06 GMT
Hi Caleb,

The logs for a Fluo application can be found in YARN but they are tricky to
find. Fluo should have better documentation on this which I will add now.

The easiest way to view the logs for a Fluo application is to use the web
interface for the YARN resource manager (http://localhost:8088/cluster).
First, click on the application ID (i.e application_*) of your Fluo
application and then click on the latest attempt ID (appattempt_*). You
should see a list of containers.  There should be a container for the
application master (typically container 1), a Fluo oracle (typically
container 2), and Fluo workers (containers 3+).  You can view the log files
produced by a container by clicking on its 'logs' link.  Logs from Fluo
observers will be in the worker_*.log file for each of your worker
containers.

If you don't want to use the YARN resource manager web interface, you can
also view these logs in the directory specified by
'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config.  This method
works well on one machine but on cluster your containers will probably be
on different machines. See the YARN documentation below for more info about
this property:

https://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

Best,
Mike

On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb <Caleb.Meier@parsons.com> wrote:

Hello,

I'm attempting to debug a Fluo application and am having difficulty
locating the logs for my observers.  I've looked within the logs for
hadoop-yarn, but am not seeing any logging statements for my observers.
Where do observers log out of the box in a normal cloudera distribution?
Do I need to do something else in addition to logging to get my observers
to generate logs?

Thanks,
Caleb

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message