airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niels Zeilemaker <ni...@zeilemaker.nl>
Subject Re: DAG logging
Date Tue, 31 Oct 2017 19:59:50 GMT
How would I access the logging from within a PyhtonOperator python callable?

That's a method that's defined in your dag, but doesn't have a reference to
the operator.

Niels

Op 31 okt. 2017 20:56 schreef "Bolke de Bruin" <bdbruin@gmail.com>:

> Where do you want those to end up? As they are (probably) evaluated during
> parsing, they will end up in the log of the parsing process. So dag
> processor log file or executor (celery worker).
>
> Bolke
>
> > On 31 Oct 2017, at 20:31, Chris Riccomini <criccomini@apache.org> wrote:
> >
> > How does this work for DAG logging (as opposed to task logging). DAG
> > logging can't easily use LoggingMixin. Is there some example code
> somewhere
> > about what to do on DAGs?
> >
> > On Tue, Oct 31, 2017 at 11:22 AM, Boris Tyukin <boris@boristyukin.com>
> > wrote:
> >
> >> Chris,
> >>
> >> see my post "new logging" - apparently we cannot use logging any more
> and
> >> have to init log handler.
> >>
> >> On Tue, Oct 31, 2017 at 1:54 PM, Chris Riccomini <criccomini@apache.org
> >
> >> wrote:
> >>
> >>> Correction:
> >>>
> >>> import logging
> >>>
> >>> class DqRowCheckOperator(BaseOperator):
> >>>  ...
> >>>  def execute(...):
> >>>    logging.info('foo')
> >>>  ...
> >>>
> >>> It's an operator that we're using. The 'foo' doesn't show up in the
> logs
> >> in
> >>> the UI or file.
> >>>
> >>> On Tue, Oct 31, 2017 at 10:47 AM, Chris Riccomini <
> criccomini@apache.org
> >>>
> >>> wrote:
> >>>
> >>>> Hey all,
> >>>>
> >>>> Just noticed when we upgraded to 1.9.0 that logging from our custom
> >>>> operators are no longer visible in the file. Assuming this is due to
> >> all
> >>>> the log changes that were made in 1.9.0.
> >>>>
> >>>> Our custom operators just have:
> >>>>
> >>>> import logging
> >>>>
> >>>> class DbDagBuilder(object):
> >>>>  ...
> >>>>  logging.info('foo')
> >>>>  ...
> >>>>
> >>>> This was working fine in 1.8.2. What is the suggested way to make this
> >>>> work?
> >>>>
> >>>> Cheers,
> >>>> Chris
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message