airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Standish <dpstand...@gmail.com>
Subject Re: scheduler logging
Date Tue, 23 Apr 2019 18:38:57 GMT
I noticed two issues in 1.10.2, one of which has been resolved, but I am
not sure if they were present in 1.10.0, or if they are the messages that
are bothering you.

1. "Harvesting DAG parsing results" was printed every 2 seconds or
something.
This was resolved in commit [AIRFLOW-3911] Change Harvesting DAG parsing
results to DEBUG log level (#4729) which I believe is in 1.10.3 now.
It just changes log level to debug for that message.

2. Frequent "db connection invalidated" warning
This one is unsolved.  I created a ticket here:
https://issues.apache.org/jira/browse/AIRFLOW-4134
It seems like every 5 seconds the warning "db connection invalidated" is
logged.
It happens in a connection reconnect try loop.  It always seems to be able
to reconnect on first try, so one idea is we could just set it to only warn
if the first retry fails (debug on first reconnect).  But it would be more
satisfying to figure out why this connection always seems to be invalidated
and fix the root cause.  Alas I am not sure how to proceed...



On Tue, Apr 23, 2019 at 9:54 AM Bolke de Bruin <bdbruin@gmail.com> wrote:

> It's probably better to create a custom logging.conf and use that instead.
>
> B.
>
> Op di 23 apr. 2019 18:13 schreef Frank Maritato
> <fmaritato@opentable.com.invalid>:
>
> > No one else has this issue? Or no one has a solution?
> >
> >
> > On Wed, Apr 17, 2019 at 5:49 PM Frank Maritato <fmaritato@opentable.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > We are running airflow 1.10.0 and I'm wondering how I can turn off or
> > turn
> > > down the logging for the scheduler? I tried setting logging_level=WARN
> in
> > > airflow.cfg and restarting the process but I'm still seeing a ton of
> info
> > > logging to .out and .log.
> > >
> > > Thanks!
> > > --
> > > Frank Maritato
> > >
> >
> >
> > --
> > Frank Maritato
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message