lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Configure writer to write to FSDirectory?
Date Sun, 05 Feb 2012 14:22:33 GMT
Are you using near-real-time readers?

(IndexReader.open(IndexWriter))

Mike McCandless

http://blog.mikemccandless.com

On Sun, Feb 5, 2012 at 9:03 AM, Cheng <zhoucheng2008@gmail.com> wrote:
> Hi Uwe,
>
> My challenge is that I need to update/modify the indexes frequently while
> providing the search capability. I was trying to use FSDirectory, but found
> out that the reading and writing from/to FSDirectory is unbearably slow. So
> I now am trying the RAMDirectory, which is fast.
>
> I don't know of  MMapDirectory, and wonder if it is as fast as RAMDirectory.
>
>
> On Sun, Feb 5, 2012 at 4:14 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
>
>> Hi Cheng,
>>
>> It seems that you use a RAMDirectory for *caching*, otherwise it makes no
>> sense to write changes back. In recent Lucene versions, this is not a good
>> idea, especially for large indexes (RAMDirectory eats your heap space,
>> allocates millions of small byte[] arrays,...). If you need something like
>> a
>> caching Directory and you are working on a 64bit platform, you can use
>> MMapDirectory (where the operating system kernel manages the read/write
>> between disk an memory). MMapDirectory is returned by default for
>> FSDirectory.open() on most 64 bit platforms. The good thing: the "caching"
>> space is outside your JVM heap, so does not slowdown the garbage collector.
>> So be sure to *not* allocate too much heap space (-Xmx) to your search app,
>> only the minimum needed to execute it and leave the rest of your RAM
>> available for the OS kernel to manage FS cache.
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>> > -----Original Message-----
>> > From: Cheng [mailto:zhoucheng2008@gmail.com]
>> > Sent: Sunday, February 05, 2012 7:56 AM
>> > To: java-user@lucene.apache.org
>> > Subject: Configure writer to write to FSDirectory?
>> >
>> > Hi,
>> >
>> > I build an RAMDirectory on a FSDirectory, and would like the writer
>> associated
>> > with the RAMDirectory to periodically write to hard drive.
>> >
>> > Is this achievable?
>> >
>> > Thanks.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message