lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: High disk write usage
Date Wed, 05 Jul 2017 17:03:57 GMT
bq: We have enough physical RAM to store full collection and 16Gb for each JVM.

That's not quite what I was asking for. Lucene uses MMapDirectory to
map part of the index into the OS memory space. If you've
over-allocated the JVM space relative to your physical memory that
space can start swapping. Frankly I'd expect your query performance to
die if that was happening so this is a sanity check.

How much physical memory does the machine have and how much memory is
allocated to _all_ of the JVMs running on that machine?

see: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

Best,
Erick


On Wed, Jul 5, 2017 at 9:41 AM, Antonio De Miguel <devetonez@gmail.com> wrote:
> Hi Erik! thanks for your response!
>
> Our soft commit is 5 seconds. Why generates I/0 a softcommit? first notice.
>
>
> We have enough physical RAM to store full collection and 16Gb for each
> JVM.  The collection is relatively small.
>
> I've tried (for testing purposes)  disabling transactionlog (commenting
> <updateLog>)... but cluster does not go up. I'll try writing into separated
> drive, nice idea...
>
>
>
>
>
>
>
>
> 2017-07-05 18:04 GMT+02:00 Erick Erickson <erickerickson@gmail.com>:
>
>> What is your soft commit interval? That'll cause I/O as well.
>>
>> How much physical RAM and how much is dedicated to _all_ the JVMs on a
>> machine? One cause here is that Lucene uses MMapDirectory which can be
>> starved for OS memory if you use too much JVM, my rule of thumb is
>> that _at least_ half of the physical memory should be reserved for the
>> OS.
>>
>> Your transaction logs should fluctuate but even out. By that I mean
>> they should increase in size but every hard commit should truncate
>> some of them so I wouldn't expect them to grow indefinitely.
>>
>> One strategy is to put your tlogs on a separate drive exactly to
>> reduce contention. You could disable them too at a cost of risking
>> your data. That might be a quick experiment you could run though,
>> disable tlogs and see what that changes. Of course I'd do this on my
>> test system ;).
>>
>> But yeah, Solr will use a lot of I/O in the scenario you are outlining
>> I'm afraid.
>>
>> Best,
>> Erick
>>
>> On Wed, Jul 5, 2017 at 8:08 AM, Antonio De Miguel <devetonez@gmail.com>
>> wrote:
>> > thanks Markus!
>> >
>> > We already have SSD.
>> >
>> > About changing topology.... we probed yesterday with 10 shards, but
>> system
>> > goes more inconsistent than with the current topology (5x10). I dont know
>> > why... too many traffic perhaps?
>> >
>> > About merge factor.. we set default configuration for some days... but
>> when
>> > a merge occurs system overload. We probed with mergefactor of 4 to
>> improbe
>> > query times and trying to have smaller merges.
>> >
>> > 2017-07-05 16:51 GMT+02:00 Markus Jelsma <markus.jelsma@openindex.io>:
>> >
>> >> Try mergeFactor of 10 (default) which should be fine in most cases. If
>> you
>> >> got an extreme case, either create more shards and consider better
>> hardware
>> >> (SSD's)
>> >>
>> >> -----Original message-----
>> >> > From:Antonio De Miguel <devetonez@gmail.com>
>> >> > Sent: Wednesday 5th July 2017 16:48
>> >> > To: solr-user@lucene.apache.org
>> >> > Subject: Re: High disk write usage
>> >> >
>> >> > Thnaks a lot alessandro!
>> >> >
>> >> > Yes, we have very big physical dedicated machines, with a topology
of
>> 5
>> >> > shards and10 replicas each shard.
>> >> >
>> >> >
>> >> > 1. transaction log files are increasing but not with this rate
>> >> >
>> >> > 2.  we 've probed with values between 300 and 2000 MB... without any
>> >> > visible results
>> >> >
>> >> > 3.  We don't use those features
>> >> >
>> >> > 4. No.
>> >> >
>> >> > 5. I've probed with low and high mergefacors and i think that is  the
>> >> point.
>> >> >
>> >> > With low merge factor (over 4) we 've high write disk rate as i said
>> >> > previously
>> >> >
>> >> > with merge factor of 20, writing disk rate is decreasing, but now,
>> with
>> >> > high qps rates (over 1000 qps) system is overloaded.
>> >> >
>> >> > i think that's the expected behaviour :(
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > 2017-07-05 15:49 GMT+02:00 alessandro.benedetti <a.benedetti@sease.io
>> >:
>> >> >
>> >> > > Point 2 was the ram Buffer size :
>> >> > >
>> >> > > *ramBufferSizeMB* sets the amount of RAM that may be used by Lucene
>> >> > >          indexing for buffering added documents and deletions
before
>> >> they
>> >> > > are
>> >> > >          flushed to the Directory.
>> >> > >          maxBufferedDocs sets a limit on the number of documents
>> >> buffered
>> >> > >          before flushing.
>> >> > >          If both ramBufferSizeMB and maxBufferedDocs is set, then
>> >> > >          Lucene will flush based on whichever limit is hit first.
>> >> > >
>> >> > > <ramBufferSizeMB>100</ramBufferSizeMB>
>> >> > > <maxBufferedDocs>1000</maxBufferedDocs>
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > -----
>> >> > > ---------------
>> >> > > Alessandro Benedetti
>> >> > > Search Consultant, R&D Software Engineer, Director
>> >> > > Sease Ltd. - www.sease.io
>> >> > > --
>> >> > > View this message in context: http://lucene.472066.n3.
>> >> > > nabble.com/High-disk-write-usage-tp4344356p4344386.html
>> >> > > Sent from the Solr - User mailing list archive at Nabble.com.
>> >> > >
>> >> >
>> >>
>>

Mime
View raw message