lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Bazhenov <dot...@gmail.com>
Subject Re: RAM or SSD...
Date Fri, 27 Jul 2012 22:51:59 GMT
Another option is to use tmpfs (memory driven file system) as a backing storage for index.
This allows to minimize disc access latency. In average case this doesn't make sense, because
OS will cache file system access quite effective if you have enough memory. But in worst case
(if you measure response time 99 percentile or higher, for example) this method can give you
more stable performance.

On Jul 20, 2012, at 10:32 PM, Dragon Fly <dragon-fly999@hotmail.com> wrote:

> 
> Thank you.
> 
>> From: dawid.weiss@gmail.com
>> Date: Thu, 19 Jul 2012 13:34:26 +0200
>> Subject: Re: RAM or SSD...
>> To: java-user@lucene.apache.org
>> 
>> Read this:
>> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>> 
>> Dawid
>> 
>> On Thu, Jul 19, 2012 at 1:32 PM, Dragon Fly <dragon-fly999@hotmail.com> wrote:
>>> 
>>> The slowest part of my application is to read the search hits from disk.  I was
hoping that using an SSD or RAMDirectory/MMapDirectory would speed that up.  I read the JavaDoc
for MMapDirectory but didn't really understand how that differs from RAMDirectory.  Could
someone please explain?
>>> 
>>>> Date: Wed, 18 Jul 2012 23:46:51 -0700
>>>> Subject: Re: RAM or SSD...
>>>> From: vfunstein@gmail.com
>>>> To: java-user@lucene.apache.org
>>>> 
>>>> I was referring to *RAMDirectory*.
>>>> 
>>>> On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <goksron@gmail.com>
wrote:
>>>>>> You do not want to store 30 G of data in the JVM heap, no matter
what
>>>> library does this.
>>>>> MMapDirectory does not store data in the JVM heap. It lets the
>>>>> operating system manage the disk buffer space. Even if the JVM says "I
>>>>> have 30G of memory space", it really does not. It only has address
>>>>> space allocated by the  OS but no memory.
>>>>> 
>>>>> On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te@statsbiblioteket.dk>
>>>> wrote:
>>>>>> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
>>>>>>> If I want to improve performance, which of the following is better
and
>>>> why?
>>>>>>> 
>>>>>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for
the index.
>>>>>> 
>>>>>> As others has pointed out, MMapDirectory should work better than
>>>>>> RAMDirectory. I am sure it will work fine with a relative small index
>>>>>> such as yours. However, it does not scale that well with index size.
>>>>>> 
>>>>>>> 2. Put the index on a solid state drive.
>>>>>> 
>>>>>> Why anyone buys computers without SSD's is a mystery to me. Use SSDs
for
>>>>>> the small low-latency stuff and a secondary spinning drive for the
large
>>>>>> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls
into
>>>>>> the small low-latency bucket. SSDs speeds up almost everything, saves
>>>>>> RAM and spares a lot of work hours optimizing I/O-speed.
>>>>>> 
>>>>>> Regards,
>>>>>> Toke Eskildsen
>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Lance Norskog
>>>>> goksron@gmail.com
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>> 
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
> 		 	   		  

---
Denis Bazhenov <dotsid@gmail.com>






---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message