lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <>
Subject Re: Realtime Search
Date Wed, 24 Dec 2008 05:17:40 GMT
Also, if you are thinking that accessing the "buffer" directly will  
be faster than "parsing" the packed structure, I'm not so sure.

You can review the source for the various buffers, and since the is  
no "struct" support in Java, you end up combining bytes to make  
longs, etc. Also, a lot of the accesses are though Unsafe, which is  
slower than the indirection on a Java object to access a field.

My review of these classes makes me think that parsing the "skip to"  
index once into java objects for later use is going to be a lot  
faster overall than accessing the entire mapped file directly on  
every invocation.

On Dec 23, 2008, at 9:20 PM, Marvin Humphrey wrote:

> On Tue, Dec 23, 2008 at 08:36:24PM -0600, robert engels wrote:
>> Is there something that I am missing?
> Yes.
>> I see lots of references to  using "memory mapped" files to  
>> "dramatically"
>> improve performance.
> There have been substantial discussions about this design in JIRA,
> notably LUCENE-1458.
> The "dramatic" improvement is WRT to opening/reopening an IndexReader.
> Presently in both KS and Lucene, certain data structures have to be  
> read at
> IndexReader startup and unpacked into process memory -- in  
> particular, the
> term dictionary index and sort caches.  If those data structures  
> can be
> represented by a memory mapped file rather than built up from  
> scratch, we save
> big.
> Marvin Humphrey
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message