lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: FST and FieldCache?
Date Thu, 19 May 2011 16:43:04 GMT
On Thu, May 19, 2011 at 12:35 PM, Jason Rutherglen
<jason.rutherglen@gmail.com> wrote:
>> And I do agree there are times when mmap is appropriate, eg if query
>> latency is unimportant to you, but it's not a panacea and it comes
>> with serious downsides
>
> Do we have a benchmark of ByteBuffer vs. byte[]'s in RAM?

I don't know of a straight up comparison...

> There's also RAM based SSDs whose performance could be comparable with
> well, RAM.

True, though it's through layers of abstraction designed originally
for serving files off of spinning magnets :)

> Also, with our heap based field caches, the first sorted
> search requires that they be loaded into RAM.  Then we don't unload
> them until the reader is closed?  With MMap the unloading would happen
> automatically?

True, but really if the app knows it won't need that FC entry for a
long time (ie, long enough to make it worth unloading/reloading) then
it should really unload it.  MMap would still have to write all those
pages to disk...

DocValues actually makes this a lot cheaper because loading DocValues
is much (like ~100 X from Simon's testing) faster than populating
FieldCache since FieldCache must do all the uninverting.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message