lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Solanki <nitinml...@gmail.com>
Subject Re: Solr Logging files get high
Date Wed, 04 Feb 2015 06:27:53 GMT
Thanks Michael Della Bitta.

Hi. Mike Sokolov,
                        There is no DEBUG appears inside logs..

On Tue, Feb 3, 2015 at 10:06 PM, Michael Della Bitta <
michael.della.bitta@appinions.com> wrote:

> If you're trying to do a bulk ingest of data, I recommend committing less
> frequently. Don't soft commit at all until the end of the batch, and hard
> commit every 60 seconds.
>
> Michael Della Bitta
>
> Senior Software Engineer
>
> o: +1 646 532 3062
>
> appinions inc.
>
> “The Science of Influence Marketing”
>
> 18 East 41st Street
>
> New York, NY 10017
>
> t: @appinions <https://twitter.com/Appinions> | g+:
> plus.google.com/appinions
> <
> https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts
> >
> w: appinions.com <http://www.appinions.com/>
>
> On Tue, Feb 3, 2015 at 12:51 AM, Nitin Solanki <nitinmlvya@gmail.com>
> wrote:
>
> > Hi Michael Della and Michael Sokolov,
> >
> > *size of tlog :-*
> > 56K    /mnt/nitin/solr/node1/solr/wikingram_shard3_replica1/data/tlog/
> > 56K    /mnt/nitin/solr/node1/solr/wikingram_shard7_replica1/data/tlog/
> > 56K    /mnt/nitin/solr/node2/solr/wikingram_shard4_replica1/data/tlog/
> > 52K    /mnt/nitin/solr/node2/solr/wikingram_shard8_replica1/data/tlog/
> > 52K    /mnt/nitin/solr/node3/solr/wikingram_shard1_replica1/data/tlog/
> > 52K    /mnt/nitin/solr/node3/solr/wikingram_shard5_replica1/data/tlog/
> > 56K    /mnt/nitin/solr/node4/solr/wikingram_shard2_replica1/data/tlog/
> > 48K    /mnt/nitin/solr/node4/solr/wikingram_shard6_replica1/data/tlog/
> >
> > *Size of logs :-*
> > 755M    /mnt/nitin/solr/node1/logs/
> > 729M    /mnt/nitin/solr/node2/logs/
> > 729M    /mnt/nitin/solr/node3/logs/
> > 729M    /mnt/nitin/solr/node4/logs/
> >
> > Which log is reducing performance?  I am committing more frequent hard
> > commits. After 1 second , I am performing soft commit and after 15
> seconds,
> > I am performing hard commit. I indexed 2 GB of data and you can see the
> > size of tlog that I pasted above. Is this tlog is good for 2GB indexed
> > data? Or is it high? The main question is that size of log will harm
> > performance of Solr?
> >
> >
> >
> >
> > On Mon, Feb 2, 2015 at 10:27 PM, Michael Della Bitta <
> > michael.della.bitta@appinions.com> wrote:
> >
> > > Good call, it could easily be the tlog Nitin is talking about.
> > >
> > > As for which definition of high, I was making assumptions as well. :)
> > >
> > > Michael Della Bitta
> > >
> > > Senior Software Engineer
> > >
> > > o: +1 646 532 3062
> > >
> > > appinions inc.
> > >
> > > “The Science of Influence Marketing”
> > >
> > > 18 East 41st Street
> > >
> > > New York, NY 10017
> > >
> > > t: @appinions <https://twitter.com/Appinions> | g+:
> > > plus.google.com/appinions
> > > <
> > >
> >
> https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts
> > > >
> > > w: appinions.com <http://www.appinions.com/>
> > >
> > > On Mon, Feb 2, 2015 at 11:51 AM, Michael Sokolov <
> > > msokolov@safaribooksonline.com> wrote:
> > >
> > > > I was tempted to suggest rehab -- but seriously it wasn't clear if
> > Nitin
> > > > meant the log files Michael is referring to, or the transaction log
> > > > (tlog).  If it's the transaction log, the solution is more frequent
> > hard
> > > > commits.
> > > >
> > > > -Mike
> > > >
> > > > On 2/2/2015 11:48 AM, Michael Della Bitta wrote:
> > > >
> > > >> If you'd like to reduce the amount of lines Solr logs, you need to
> > edit
> > > >> the
> > > >> file example/resources/log4j.properties in Solr's home directory.
> > Change
> > > >> lines that say INFO to WARN.
> > > >>
> > > >> Michael Della Bitta
> > > >>
> > > >> Senior Software Engineer
> > > >>
> > > >> o: +1 646 532 3062
> > > >>
> > > >> appinions inc.
> > > >>
> > > >> “The Science of Influence Marketing”
> > > >>
> > > >> 18 East 41st Street
> > > >>
> > > >> New York, NY 10017
> > > >>
> > > >> t: @appinions <https://twitter.com/Appinions> | g+:
> > > >> plus.google.com/appinions
> > > >> <https://plus.google.com/u/0/b/112002776285509593336/
> > > >> 112002776285509593336/posts>
> > > >> w: appinions.com <http://www.appinions.com/>
> > > >>
> > > >> On Mon, Feb 2, 2015 at 7:42 AM, Nitin Solanki <nitinmlvya@gmail.com
> >
> > > >> wrote:
> > > >>
> > > >>  Hi,
> > > >>>           My solr logs directory has been get high. It is seriously
> > > >>> problem
> > > >>> or It harms my solr performance in both cases indexing as well
as
> > > >>> searching.
> > > >>>
> > > >>>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message