Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Subject: Re: CDCR logging is Needlessly verbose, fills up the file system fast
To: solr-user@lucene.apache.org
References: <CAKTEqfk4H8txz9YhoUcHFwc+87+681SYdPDUG-RYWH9PEd4fsQ@mail.gmail.com>
 <CAKTEqf=FF1s4bL-9hM-PQZ1-Vg7wN687C3SawJcbtRFxqYLjGA@mail.gmail.com>
 <CAN4YXveg60fY8LVr+U0Xoz+n-6V=ZmUYoRHDVK5GAtO3c_XpiQ@mail.gmail.com>
 <CAKTEqfkDwLELAVhEb-tHaXZ+TsBdVQaa-e=ctqiBSAJPFWh+=Q@mail.gmail.com>
 <D0B72534-A6EC-4C0B-9B90-0D4D9A36A933@flax.co.uk>
 <CAKTEqf=VMPGtWe-jGNsvutZdL2F3U+2L56j2DT20wWBpsr3_2A@mail.gmail.com>
 <6b3168cf-ac67-9d86-3e64-2fa829f63dcf@elyograg.org>
 <CAKTEqfm0-ODMXR5hS63N1Tn87vC36tOEd69g_R7KziAuMuaYiw@mail.gmail.com>
From: Shawn Heisey <apache@elyograg.org>
Message-ID: <08848ed0-0b88-d7a8-2efb-12a3f84383e7@elyograg.org>
Date: Fri, 6 Jan 2017 08:52:06 -0700
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.6.0
MIME-Version: 1.0
In-Reply-To: <CAKTEqfm0-ODMXR5hS63N1Tn87vC36tOEd69g_R7KziAuMuaYiw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 06 Jan 2017 15:52:41 -0000

On 1/6/2017 8:21 AM, Webster Homer wrote:
> I figured our problem with the filesystem, by default the root logger
> is configured with the CONSOLE logger, which is NOT rotated and
> eventually filled up the file system. That doesn't exonerate the CDCR
> logging problem though. The thing writes a huge amount of junk to the
> logs, information that looks more like Debug and even TRACE level
> statements. I hope that this is addressed, and soon! There doesn't
> seem to be a way to turn it off either. We want INFO level logging

The logging that goes to the console and solr.log by default is INFO. 
Even without CDCR, INFO logging is fairly verbose, but normally doesn't
chew up a whole bunch of gigabytes over a short timeframe.

CDCR is a pretty new feature, and the authors likely needed a lot of
information in the logs while they were developing it so they could
chase down bugs.  I have never used the feature and haven't looked at
the logging, but it sounds like much of it could be changed to DEBUG or
lower priority now that primary development is done.  If you haven't
already opened an issue in Jira to reduce what CDCR logs at INFO, that
would be a good idea.

To get rid of the console logging entirely, which is a good idea, edit
server/resources/log4j.properties.  Change the "log4j.rootLogger" line
to remove CONSOLE and the comma.  Optionally, you can also remove all
the other lines that contain CONSOLE.

Starting in version 6.3, you can put "-Dsolr.log.muteconsole" on the
java commandline (normally you'd do this in the solr.in.* script) to
accomplish this without changing the logging config.  I personally think
that Solr shouldn't log to the console at all unless it is running in
the foreground and the console isn't being redirected to a file.

https://issues.apache.org/jira/browse/SOLR-8186

Thanks,
Shawn