lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng <zhoucheng2...@gmail.com>
Subject Re: Configure writer to write to FSDirectory?
Date Mon, 06 Feb 2012 00:15:54 GMT
I was trying to, but don't know how to even I read some of your blogs.

On Sun, Feb 5, 2012 at 10:22 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> Are you using near-real-time readers?
>
> (IndexReader.open(IndexWriter))
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Sun, Feb 5, 2012 at 9:03 AM, Cheng <zhoucheng2008@gmail.com> wrote:
> > Hi Uwe,
> >
> > My challenge is that I need to update/modify the indexes frequently while
> > providing the search capability. I was trying to use FSDirectory, but
> found
> > out that the reading and writing from/to FSDirectory is unbearably slow.
> So
> > I now am trying the RAMDirectory, which is fast.
> >
> > I don't know of  MMapDirectory, and wonder if it is as fast as
> RAMDirectory.
> >
> >
> > On Sun, Feb 5, 2012 at 4:14 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> >> Hi Cheng,
> >>
> >> It seems that you use a RAMDirectory for *caching*, otherwise it makes
> no
> >> sense to write changes back. In recent Lucene versions, this is not a
> good
> >> idea, especially for large indexes (RAMDirectory eats your heap space,
> >> allocates millions of small byte[] arrays,...). If you need something
> like
> >> a
> >> caching Directory and you are working on a 64bit platform, you can use
> >> MMapDirectory (where the operating system kernel manages the read/write
> >> between disk an memory). MMapDirectory is returned by default for
> >> FSDirectory.open() on most 64 bit platforms. The good thing: the
> "caching"
> >> space is outside your JVM heap, so does not slowdown the garbage
> collector.
> >> So be sure to *not* allocate too much heap space (-Xmx) to your search
> app,
> >> only the minimum needed to execute it and leave the rest of your RAM
> >> available for the OS kernel to manage FS cache.
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >> > -----Original Message-----
> >> > From: Cheng [mailto:zhoucheng2008@gmail.com]
> >> > Sent: Sunday, February 05, 2012 7:56 AM
> >> > To: java-user@lucene.apache.org
> >> > Subject: Configure writer to write to FSDirectory?
> >> >
> >> > Hi,
> >> >
> >> > I build an RAMDirectory on a FSDirectory, and would like the writer
> >> associated
> >> > with the RAMDirectory to periodically write to hard drive.
> >> >
> >> > Is this achievable?
> >> >
> >> > Thanks.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message