fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Meier, Caleb" <Caleb.Me...@parsons.com>
Subject RE: debugging fluo
Date Tue, 01 Nov 2016 20:22:34 GMT
Hey Mike,

So I’m not seeing any worker logs on my machines.  The search

find / | grep worker

yielded nothing.  Any ideas as to why these don't exist?  I'm not sure
that my fluo.log.dir system property is successfully being set by the LogbackUtil class.
Maybe this has something to do with it?


-----Original Message-----
From: Mike Walch [mailto:mwalch@apache.org] 
Sent: Tuesday, November 01, 2016 4:03 PM
To: dev@fluo.incubator.apache.org
Subject: Re: debugging fluo

Were you able to find a worker_*.log file for each of your workers?

Below are some tips for debugging:

- Each YARN container should have a 'stdout' and 'stderr' file.  These files may have helpful
error messages.  Especially if a worker failed to start.  Also, any calls to System.out and
System.err in your observer will be printed to these files.
- When running Fluo in YARN, Fluo must use Logback for logging (due to a hard requirment by
Twill). Logback is configured using /path/to/fluo/conf/logback.xml.  You should review this
configuration but the root logger is configured by default to print any message that is the
debug level or higher.
- If you configured multiple workers, each worker will run in a different container and have
a different worker_*.log file.
- When a worker starts up, it prints its configuration to worker_*.log.
Make sure that you configured your observers using the property 'fluo.observer.*'


On Tue, Nov 1, 2016 at 3:33 PM Meier, Caleb <Caleb.Meier@parsons.com> wrote:

> Do you have any tips for how to make Observers log to the log files 
> found in the directory specified by 'yarn.nodemanager.log-dirs'?
> -----Original Message-----
> From: Mike Walch [mailto:mwalch@apache.org]
> Sent: Tuesday, November 01, 2016 2:36 PM
> To: dev@fluo.incubator.apache.org
> Subject: Re: debugging fluo
> Hi Caleb,
> The logs for a Fluo application can be found in YARN but they are 
> tricky to find. Fluo should have better documentation on this which I will add now.
> The easiest way to view the logs for a Fluo application is to use the 
> web interface for the YARN resource manager ( 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8088_c
> luster&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzY
> C2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6
> mq5Tns5Aewd2s&s=_4PS5z_vu1bkhpZBdzJXjbGsCvMBboMqoLBIOBnRAEY&e=
> ).
> First, click on the application ID (i.e application_*) of your Fluo 
> application and then click on the latest attempt ID (appattempt_*). 
> You should see a list of containers.  There should be a container for 
> the application master (typically container 1), a Fluo oracle 
> (typically container 2), and Fluo workers (containers 3+).  You can 
> view the log files produced by a container by clicking on its 'logs' 
> link.  Logs from Fluo observers will be in the worker_*.log file for 
> each of your worker containers.
> If you don't want to use the YARN resource manager web interface, you 
> can also view these logs in the directory specified by 
> 'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config.  This 
> method works well on one machine but on cluster your containers will 
> probably be on different machines. See the YARN documentation below 
> for more info about this property:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__hadoop.apache.org
> _docs_r2.7.0_hadoop-2Dyarn_hadoop-2Dyarn-2Dcommon_yarn-2Ddefault.xml&d
> =CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZR
> 5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5A
> ewd2s&s=SS8YTOgIAWUmBnKkHN2Eu2-h6WyEHqlNvJO-D5EKFiI&e=
> Best,
> Mike
> On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb <Caleb.Meier@parsons.com>
> wrote:
> Hello,
> I'm attempting to debug a Fluo application and am having difficulty 
> locating the logs for my observers.  I've looked within the logs for 
> hadoop-yarn, but am not seeing any logging statements for my observers.
> Where do observers log out of the box in a normal cloudera distribution?
> Do I need to do something else in addition to logging to get my 
> observers to generate logs?
> Thanks,
> Caleb
View raw message