lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng <zhoucheng2...@gmail.com>
Subject Re: RAMDirectory unexpectedly slows
Date Mon, 04 Jun 2012 14:55:11 GMT
My indexes are 500MB+. So it seems like that RAMDirectory is not good for
that big a size.

My challenge, on the other side, is that I need to update the indexes very
frequently. So, do you think  MMapDirectory is the solution?

Thanks.

On Mon, Jun 4, 2012 at 10:30 PM, Jack Krupansky <jack@basetechnology.com>wrote:

> From the javadoc for RAMDirectory:
>
> "Warning: This class is not intended to work with huge indexes. Everything
> beyond several hundred megabytes will waste resources (GC cycles), because
> it uses an internal buffer size of 1024 bytes, producing millions of
> byte[1024] arrays. This class is optimized for small memory-resident
> indexes. It also has bad concurrency on multithreaded environments.
>
> It is recommended to materialize large indexes on disk and use
> MMapDirectory, which is a high-performance directory implementation working
> directly on the file system cache of the operating system, so copying data
> to Java heap space is not useful."
>
> -- Jack Krupansky
>
> -----Original Message----- From: Cheng
> Sent: Monday, June 04, 2012 10:08 AM
> To: java-user@lucene.apache.org
> Subject: RAMDirectory unexpectedly slows
>
>
> Hi,
>
> My apps need to read from and write to some big indexes frequently. So I
> use RAMDirectory instead of FSDirectory, and give JVM about 2GB memory
> size.
>
> I notice that the speed of reading and writing unexpectedly slows as the
> size of the indexes increases. Since the usage of RAM is less than 20%, I
> think by default the RAMDirectory doesn't take advantage of the memory I
> assigned to JVM.
>
> What are the steps to improve the reading and writing speed of
> RAMDirectory?
>
> Thanks!
> Jeff
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.**apache.org<java-user-unsubscribe@lucene.apache.org>
> For additional commands, e-mail: java-user-help@lucene.apache.**org<java-user-help@lucene.apache.org>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message