fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: debugging fluo
Date Tue, 01 Nov 2016 20:21:39 GMT
On Tue, Nov 1, 2016 at 4:02 PM, Mike Walch <mwalch@apache.org> wrote:
> Were you able to find a worker_*.log file for each of your workers?
> Below are some tips for debugging:
> - Each YARN container should have a 'stdout' and 'stderr' file.  These
> files may have helpful error messages.  Especially if a worker failed to
> start.  Also, any calls to System.out and System.err in your observer will
> be printed to these files.
> - When running Fluo in YARN, Fluo must use Logback for logging (due to a
> hard requirment by Twill). Logback is configured using
> /path/to/fluo/conf/logback.xml.  You should review this configuration but
> the root logger is configured by default to print any message that is the
> debug level or higher.

Each application has its own config.  That config gets copied from
/path/to/fluo/conf/ to /path/to/fluo/apps/$APPNAME/conf when the Fluo
app is created.  So wouldn't logging config changes need to be made to

> - If you configured multiple workers, each worker will run in a different
> container and have a different worker_*.log file.
> - When a worker starts up, it prints its configuration to worker_*.log.
> Make sure that you configured your observers using the property
> 'fluo.observer.*'
> -Mike
> On Tue, Nov 1, 2016 at 3:33 PM Meier, Caleb <Caleb.Meier@parsons.com> wrote:
>> Do you have any tips for how to make Observers log to the log files found
>> in the directory specified by 'yarn.nodemanager.log-dirs'?
>> -----Original Message-----
>> From: Mike Walch [mailto:mwalch@apache.org]
>> Sent: Tuesday, November 01, 2016 2:36 PM
>> To: dev@fluo.incubator.apache.org
>> Subject: Re: debugging fluo
>> Hi Caleb,
>> The logs for a Fluo application can be found in YARN but they are tricky
>> to find. Fluo should have better documentation on this which I will add now.
>> The easiest way to view the logs for a Fluo application is to use the web
>> interface for the YARN resource manager (
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8088_cluster&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5Aewd2s&s=_4PS5z_vu1bkhpZBdzJXjbGsCvMBboMqoLBIOBnRAEY&e=
>> ).
>> First, click on the application ID (i.e application_*) of your Fluo
>> application and then click on the latest attempt ID (appattempt_*). You
>> should see a list of containers.  There should be a container for the
>> application master (typically container 1), a Fluo oracle (typically
>> container 2), and Fluo workers (containers 3+).  You can view the log files
>> produced by a container by clicking on its 'logs' link.  Logs from Fluo
>> observers will be in the worker_*.log file for each of your worker
>> containers.
>> If you don't want to use the YARN resource manager web interface, you can
>> also view these logs in the directory specified by
>> 'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config.  This method
>> works well on one machine but on cluster your containers will probably be
>> on different machines. See the YARN documentation below for more info about
>> this property:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__hadoop.apache.org_docs_r2.7.0_hadoop-2Dyarn_hadoop-2Dyarn-2Dcommon_yarn-2Ddefault.xml&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5Aewd2s&s=SS8YTOgIAWUmBnKkHN2Eu2-h6WyEHqlNvJO-D5EKFiI&e=
>> Best,
>> Mike
>> On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb <Caleb.Meier@parsons.com>
>> wrote:
>> Hello,
>> I'm attempting to debug a Fluo application and am having difficulty
>> locating the logs for my observers.  I've looked within the logs for
>> hadoop-yarn, but am not seeing any logging statements for my observers.
>> Where do observers log out of the box in a normal cloudera distribution?
>> Do I need to do something else in addition to logging to get my observers
>> to generate logs?
>> Thanks,
>> Caleb

View raw message