fluo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Meier, Caleb" <Caleb.Me...@parsons.com>
Subject RE: debugging fluo
Date Tue, 01 Nov 2016 21:01:01 GMT
I'm using fluo-incubating.  I switched "io.fluo.log" to "fluo.log" and my worker and oracle
logs appearing in my container directories.  Still working on getting my observers to log
to my worker logs though.

-----Original Message-----
From: Mike Walch [mailto:mwalch@apache.org] 
Sent: Tuesday, November 01, 2016 5:00 PM
To: dev@fluo.incubator.apache.org
Subject: Re: debugging fluo

Which version of Fluo are you using?  The properties 'io.fluo.log.*' were used before Apache
incubation in 1.0.0-beta-1 or 1.0.0-beta-2.  These properties were renamed in Fluo 1.0.0 to
'fluo.log.*'.  You could be having problems if you built your application using Fluo 1.0.0-beta-2
and ran it using a Fluo 1.0.0 install.

On Tue, Nov 1, 2016 at 4:45 PM Meier, Caleb <Caleb.Meier@parsons.com> wrote:

> Hey Keith,
>
> Not seeing the worker logs in my container.  Think my logback is 
> configured incorrectly...
>
> [root@c190sv193 container_1476563020088_0069_01_000002]# grep Rolling 
> stdout
> 18:36:49,174 |-INFO in ch.qos.logback.core.joran.action.AppenderAction 
> - About to instantiate appender of type 
> [ch.qos.logback.core.rolling.RollingFileAppender]
> 18:36:49,215 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@5ab860f4 - No 
> compression will be used
> 18:36:49,230 |-INFO in
> ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log 
> file
> name:
> io.fluo.log.dir_IS_UNDEFINED/io.fluo.log.app_IS_UNDEFINED_io.fluo.log.
> host_IS_UNDEFINED.log
> 18:36:49,230 |-INFO in
> ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property 
> is set to 
> [io.fluo.log.dir_IS_UNDEFINED/io.fluo.log.app_IS_UNDEFINED_io.fluo.log
> .host_IS_UNDEFINED.log]
>
> -----Original Message-----
> From: Keith Turner [mailto:keith@deenlo.com]
> Sent: Tuesday, November 01, 2016 4:33 PM
> To: dev@fluo.incubator.apache.org
> Subject: Re: debugging fluo
>
> Caleb,
>
> I just ran Fluo locally with Uno.  Below is some info I am seeing.
> Do you see anything about RollingFileAppender in the stdout file?
>
> $ pwd
>
> /home/kturner/uno/install/logs/yarn/application_1478030941494_0001/con
> tainer_1478030941494_0001_01_000004
> $ ls
> stderr  stdout  worker_1_host1.log
> $ grep Rolling stdout
> 16:17:31,557 |-INFO in ch.qos.logback.core.joran.action.AppenderAction
> - About to instantiate appender of type 
> [ch.qos.logback.core.rolling.RollingFileAppender]
> 16:17:31,574 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@bc1b008 - No 
> compression will be used
> 16:17:31,583 |-INFO in
> ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log 
> file
> name:
> /home/kturner/uno/install/logs/yarn/application_1478030941494_0001/con
> tainer_1478030941494_0001_01_000004/worker_1_host1
> 16:17:31,583 |-INFO in
> ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property 
> is set to 
> [/home/kturner/uno/install/logs/yarn/application_1478030941494_0001/co
> ntainer_1478030941494_0001_01_000004/worker_1_host1]
>
> Keith
>
> On Tue, Nov 1, 2016 at 4:22 PM, Meier, Caleb <Caleb.Meier@parsons.com>
> wrote:
> > Hey Mike,
> >
> > So I’m not seeing any worker logs on my machines.  The search
> >
> > find / | grep worker
> >
> > yielded nothing.  Any ideas as to why these don't exist?  I'm not 
> > sure that my fluo.log.dir system property is successfully being set 
> > by the
> LogbackUtil class.
> > Maybe this has something to do with it?
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: Mike Walch [mailto:mwalch@apache.org]
> > Sent: Tuesday, November 01, 2016 4:03 PM
> > To: dev@fluo.incubator.apache.org
> > Subject: Re: debugging fluo
> >
> > Were you able to find a worker_*.log file for each of your workers?
> >
> > Below are some tips for debugging:
> >
> > - Each YARN container should have a 'stdout' and 'stderr' file.  
> > These
> files may have helpful error messages.  Especially if a worker failed 
> to start.  Also, any calls to System.out and System.err in your 
> observer will be printed to these files.
> > - When running Fluo in YARN, Fluo must use Logback for logging (due 
> > to a
> hard requirment by Twill). Logback is configured using 
> /path/to/fluo/conf/logback.xml.  You should review this configuration 
> but the root logger is configured by default to print any message that 
> is the debug level or higher.
> > - If you configured multiple workers, each worker will run in a
> different container and have a different worker_*.log file.
> > - When a worker starts up, it prints its configuration to worker_*.log.
> > Make sure that you configured your observers using the property
> 'fluo.observer.*'
> >
> > -Mike
> >
> > On Tue, Nov 1, 2016 at 3:33 PM Meier, Caleb 
> > <Caleb.Meier@parsons.com>
> wrote:
> >
> >> Do you have any tips for how to make Observers log to the log files 
> >> found in the directory specified by 'yarn.nodemanager.log-dirs'?
> >>
> >> -----Original Message-----
> >> From: Mike Walch [mailto:mwalch@apache.org]
> >> Sent: Tuesday, November 01, 2016 2:36 PM
> >> To: dev@fluo.incubator.apache.org
> >> Subject: Re: debugging fluo
> >>
> >> Hi Caleb,
> >>
> >> The logs for a Fluo application can be found in YARN but they are 
> >> tricky to find. Fluo should have better documentation on this which 
> >> I
> will add now.
> >>
> >> The easiest way to view the logs for a Fluo application is to use 
> >> the web interface for the YARN resource manager ( 
> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A808
> >> 8_
> >> c
> >> luster&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuV
> >> dz
> >> Y
> >> C2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeE
> >> Kk
> >> 6 mq5Tns5Aewd2s&s=_4PS5z_vu1bkhpZBdzJXjbGsCvMBboMqoLBIOBnRAEY&e=
> >> ).
> >> First, click on the application ID (i.e application_*) of your Fluo 
> >> application and then click on the latest attempt ID (appattempt_*).
> >> You should see a list of containers.  There should be a container 
> >> for the application master (typically container 1), a Fluo oracle 
> >> (typically container 2), and Fluo workers (containers 3+).  You can 
> >> view the log files produced by a container by clicking on its 'logs'
> >> link.  Logs from Fluo observers will be in the worker_*.log file 
> >> for each of your worker containers.
> >>
> >> If you don't want to use the YARN resource manager web interface, 
> >> you can also view these logs in the directory specified by 
> >> 'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config.  This 
> >> method works well on one machine but on cluster your containers 
> >> will probably be on different machines. See the YARN documentation 
> >> below for more info about this property:
> >>
> >>
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__hadoop.apache.
> >> or
> >> g
> >> _docs_r2.7.0_hadoop-2Dyarn_hadoop-2Dyarn-2Dcommon_yarn-2Ddefault.xm
> >> l&
> >> d
> >> =CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kks
> >> VZ
> >> R
> >> 5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tn
> >> s5 A ewd2s&s=SS8YTOgIAWUmBnKkHN2Eu2-h6WyEHqlNvJO-D5EKFiI&e=
> >>
> >> Best,
> >> Mike
> >>
> >> On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb 
> >> <Caleb.Meier@parsons.com>
> >> wrote:
> >>
> >> Hello,
> >>
> >> I'm attempting to debug a Fluo application and am having difficulty 
> >> locating the logs for my observers.  I've looked within the logs 
> >> for hadoop-yarn, but am not seeing any logging statements for my observers.
> >> Where do observers log out of the box in a normal cloudera distribution?
> >> Do I need to do something else in addition to logging to get my 
> >> observers to generate logs?
> >>
> >> Thanks,
> >> Caleb
> >>
>
Mime
View raw message