lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4515) Make MemoryIndex more memory efficient
Date Thu, 01 Nov 2012 19:15:13 GMT


Michael McCandless commented on LUCENE-4515:

1. reduce the GC load on MemIndex if it is used extensively - that is why I moved to ByteBlockPool
/ IntBlockPool
2. control memory consumption / buffering in a very flexible way. When buffers are reused
and how many of them.

I think the current patch, minus passing allocator to MemoryIndex, solves #1 very well?

I think we should open a new Jira issue for #2 since it's apparently somewhat controversial.
 In that issue I'd really like to understand "the memory consumption / buffering in a very
flexible way / when buffers are reused and how many of them" use cases.  If it's really an
accounting issue (knowing how many bytes a given MemoryIndex is consuming) maybe we can expose
that in other ways ... eg .sizeInBytes() method, or app passes in a Counter that MemoryIndex
updates w/ how much RAM it thinks it's using, or ... something else?
> Make MemoryIndex more memory efficient
> --------------------------------------
>                 Key: LUCENE-4515
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: modules/other
>    Affects Versions: 4.0, 4.1, 5.0
>            Reporter: Simon Willnauer
>             Fix For: 4.1, 5.0
>         Attachments: LUCENE-4515.patch, LUCENE-4515.patch
> Currently MemoryIndex uses BytesRef objects to represent terms and holds an int[] per
term per field to represent postings. For highlighting this creates a ton of objects for each
search that 1. need to be GCed and 2. can't be reused.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message