lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-11934) Visit Solr logging, it's too noisy.
Date Sat, 05 May 2018 16:27:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464824#comment-16464824
] 

Shawn Heisey commented on SOLR-11934:
-------------------------------------

[~erickerickson], I like your ideas.

The query logging is also a significant contributor, *especially* on sharded indexes.  SOLR-11453
got me thinking in the background about the possibility of sending query (technically, actually
request, which includes query) logs to their own file as well.

Updates do result in particularly verbose and numerous logs.  I wonder if there's any value
to extending the 'separate logfile' paradigm for the detailed information, with the summary
info you suggested in the main log.

The only disadvantage I can think of with separate logfiles for primary logging events, and
the main notion that makes me *slightly* resistant to making that config default, is that
it is very difficult to reconstruct a narrative of events that are logged at the same millisecond
in those separate files.  If they're all in the same file, it's usually very easy to determine
the order of events.  If we do make separate request/update logs a default config, the documentation
must explain how to turn those separate logs off.  That would be vital to troubleshooting
issues that only show up under heavy load.


> Visit Solr logging, it's too noisy.
> -----------------------------------
>
>                 Key: SOLR-11934
>                 URL: https://issues.apache.org/jira/browse/SOLR-11934
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Erick Erickson
>            Assignee: Erick Erickson
>            Priority: Major
>
> I think we have way too much INFO level logging. Or, perhaps more correctly, Solr logging
needs to be examined and messages logged at an appropriate level.
> We log every update at an INFO level for instance. But I think we log LIR at INFO as
well. As a sysadmin I don't care to have my logs polluted with a message for every update,
but if I'm trying to keep my system healthy I want to see LIR messages and try to understand
why.
> Plus, in large installations logging at INFO level is creating a _LOT_ of files.
> What I want to discuss on this JIRA is
> 1> What kinds of messages do we want log at WARN, INFO, DEBUG, and TRACE levels?
> 2> Who's the audience at each level? For a running system that's functioning, sysops
folks would really like WARN messages that mean something need attention for instance. If
I'm troubleshooting should I turn on INFO? DEBUG? TRACE?
> So let's say we get some kind of agreement as to the above. Then I propose three things
> 1> Someone (and probably me but all help gratefully accepted) needs to go through
our logging and assign appropriate levels. This will take quite a while, I intend to work
on it in small chunks.
> 2> Actually answer whether unnecessary objects are created when something like log.info("whatever
{}", someObjectOrMethodCall); is invoked. Is this independent on the logging implementation
used? The SLF4J and log4j seem a bit contradictory.
> 3> Maybe regularize log, logger, LOG as variable names, but that's a nit.
> As a tactical approach, I suggest we tag each LoggerFactory.getLogger in files we work
on with //SOLR-(whatever number is assigned when I create this). We can remove them all later,
but since I expect to approach this piecemeal it'd be nice to keep track of which files have
been done already.
> Finally, I really really really don't want to do this all at once. There are 5-6 thousand
log messages. Even at 1,000 a week that's 6 weeks, even starting now it would probably span
the 7.3 release.
> This will probably be an umbrella issue so we can keep all the commits straight and people
can volunteer to "fix the files in core" as a separate piece of work (hint).
> There are several existing JIRAs about logging in general, let's link them in here as
well.
> Let the discussion begin!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message